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ABSTRACT 


This  paper  addresses  the  design  of  a  spread  spectrum  image  transmission  system 
to  provide  increased  antijam  protection  to  a  television  link  from  a  small  remotely 
piloted  vehicle.  Transform  encoding  and  frame  rate  control  may  be  used  to  re¬ 
move  the  natural  redundancy  inherent  in  the  image  in  order  to  permit  the  use  of 
spread  spectrum  encoding  for  increased  antijam  protection. 

Emphasis  is  placed  on  the  two-dimensional  discrete  cosine  transform  (DCT) 
and  a  hybrid  DCT  with  differential  pulse  code  modulation  (DPCM).  Both  the 
two-dimensional  DCT  and  the  hybrid  DCT/DPCM  provide  nearly  the  theoreti¬ 
cally  optimal  performance  of  the  Karhunen-Loeve  transform,  while  permitting 
implementation  by  small,  lightweight  hardware  in  real-time. 
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INTRODUCTION 

This  program  addresses  the  design  of  a  spread  spectrum  image  transmission  system  to 
provide  increased  anti-jam  protection  to  a  television  link  from  a  small  remotely  piloted 
vehicle.  Transform  encoding  and  frame  rate  control  may  be  used  to  remove  the  natural 
redundancy  inherent  in  the  image  in  order  to  permit  the  use  of  spread  spectrum  encoding 
for  increased  antijam  protection. 

Previous  phases  of  this  program  included  a  study  of  the  performance  of  one-  and 
two-dimensional  transforms,  the  demonstration  of  the  feasibility  of  small,  lightweight, 
real-time  implementations  of  the  discrete  Fourier  transform  (DFT),  and  the  conceptual 
design  of  similar  two-dimensional  (DFT)  implementations. 

During  the  present  reporting  period,  emphasis  has  been  placed  on  the  two-dimensional 
discrete  cosine  transform  (DCT)  and  a  hybrid  DCT  with  differential  pulse  code  modulation 
(DPCM).  Both  the  two-dimensional  DCT  and  the  hybrid  DCT/DPCM  provide  nearly  the 
theoretically  optimal  performance  of  the  Karhunen-Loeve  transform,  while  permitting 
implementation  by  small,  ’  itweight  hardware  in  real-time. 

The  conceptual  design  of  the  two-dimensional  DCT  has  been  completed,  but  its  further 
evaluation  awaits  the  development  of  a  random  access  sensor  with  nondestructive  readout. 

The  hybrid  DCT/DPCM  is  compatible  with  present  sensors  and  therefore  the  most 
promising  transform  for  use  in  the  immediate  future. 

SYSTEM  DESCRIPTION 

Two  hybrid  cosine-DPCM  bandwidth  reduction  systems  have  been  selected  for  con¬ 
struction  and  evaluation.  The  first  uses  a  Charge  Injection  Device  (CID)  [  1  ]  image  sensor 
and  a  Bucket  Brigade  Device  (BBD)  transform  implementation.  The  second  uses  an 
ordinary  vidicon  sensor  and  a  Charge  Coupled  Device  (CCD)  transform  implementation. 

In  the  CID  system  a  100x100  pixel  solid  state  sensor  will  be  used.  The  nominal 
horizontal  line  scan  will  be  one  millisecond.  The  nominal  frame  rate  will  be  10  frames  per 
second  which  can  be  displayed  without  flicker  through  the  use  of  a  scan  converter.  The 
1  millisecond  line  scan  time  was  chosen  in  order  to  match  the  sensor  to  the  BBD  filter 
which  operates  at  a  clock  rate  of  100  kHz  with  good  charge  transfer  efficiency.  At  10 


frames  per  second,  image  motion  should  be  reproduced  well  even  though  some  picture  de¬ 
tail  will  be  lost  because  of  the  low  spatial  sampling  afforded  by  the  100x100  pixel  format. 

When  the  cosine  transform  in  the  horizontal  direction  is  combined  with  Differential 
Pulse  Code  Modulation  (DPCM)  in  the  vertical  direction  to  form  a  hybrid  transform  sys¬ 
tem,  performance  essentially  indistinguishable  from  “optimum  Karhunen  Loeve  should 
be  achieved  as  shown  in  Appendix  A.  Minimum  overall  bit  rate  will  be  achieved  by  a  com¬ 
bination  of  zonal  filtering  and  variable  bit  assignment  with  low  spatial  frequencies  assigned 
more  bits  of  quantization  than  high  spatial  frequencies.  Table  1  shows  the  bit  rate  which 
results  from  three  overall  bits/pixel  assignments  at  a  pixel  rate  of  105  pixels/sec.  An  overall 
bit  assignment  of  1  bit/pixel  should  result  in  a  signal-to-distortion  ratio  of  better  than  30 
dB.  In  addition,  since  channel  errors  occur  in  the  Fourier  domain,  channel  error  rates  as 
large  as  P=10'2  will  still  provide  useful  reconstructed  images. 

The  second  bandwidth  reduction  system  will  be  compatible  with  a  standard  vidicon 
camera.  It  will  use  CCD  fillers  for  the  cosine  transform  which  will  operate  at  a  4.8  MHz 
sampling  rate.  Compatibility  with  standard  television  format  will  be  maintained  in  as 
many  aspects  as  possible.  Thus  53.5  microseconds  of  every  line  scan  will  be  used  for  video 
data  transmission  and  10  microseconds  of  every  line  will  be  reserved  for  special  functions 
such  as  synchronization.  If  the  interlace  field  is  used  directly  as  the  input  to  the  transform 
hardware  a  resolution  of  approximately  240  lines  by  256  pixels  is  possible  at  60  frames/ 
second.  This  is  equivalent  to  a  video  bandwidth  slightly  less  than  2.5  MHz.  However,  in 
order  to  reduce  the  effective  frame  rate,  a  block  of  32  adjacent  pixels  will  be  used.  Thus 
if  every  other  32  pixels  of  each  line  were  transformed,  it  would  result  in  a  horizontal  inter¬ 
lace  and  an  equivalent  frame  rate  of  30  frames/sec.  Slower  frame  rates  can  be  achieved 

Table  1.  Bit  Rate  as  a  Function  of  Quantization  for  the 
C1D  System 


Bits/Pixel 

Bit  Rate 

2 

200  kilobits/sec 

1 

1 00  kilobits/sec 

1/2 

50  kilobits/sec 

2 


by  deleting  additional  32  pixel  blocks  until  a  minimum  frame  rate  of  7.5  frames/sec  is 
achieved  when  only  one  32  pixel  block  per  line  is  used. 

The  number  of  pixels  per  frame  is  61440.  Table  2  shows  the  bit  rate  which  results  at 

several  frame  rates  and  bits/pixel. 

Table  2.  Bit  Rate  as  a  Function  of  Quantization  and  Field 
Rate  for  the  Vidicon  Compatible  System. 


Bits/Pixel 

Field  Rate 
(fields/sec) 

Bit  Rate 
(kilobits/sec) 

1 

30 

1843.2 

1 

15 

921.6 

1 

7.5 

460.8 

1/2 

30 

921.6 

1/2 

15 

460.8 

1/2 

7.5 

230.4 

SPREAD  SPECTRUM  ENCODING 

The  amount  of  anti-jam  capability  of  the  system  will  depend  on  the  subsequent  coding 
algorithm  used  to  encode  the  data.  A  relative  compression  ratio  (in  dB)  can  be  defined  for 
use  in  comparing  various  bandwidth  compression  schemes 

channel  bit  rate^ 

C.  R.  —  10  log  (  ]sj*g*p 

where  N  is  the  number  of  pixels  per  video  frame,  B  is  the  number  of  quantization  bits  per 
pixel  and  F  is  the  frame  rate.  The  channel  bit  rate  is  assumed  to  be  20.0  megabits  per 
second . 

The  compression  ratio  is,  in  a  sense,  an  approximate  lower  bound  on  the  amount  of 
anti-jam  capability  available  by  a  not  very  sophisticated  coding  scheme.  A  bit  stream  repe¬ 
tition  would  be  one  such  naive  scheme.  A  candidate  scheme  to  use  maximal  length  PN 
cyclic  codes  to  spread  the  spectrum  appears  to  offer  significant  improvements  over  naive 
schemes. 
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Consider  the  conventional  vidicon  system  in  which  it  is  planned  to  transmit  32  p  xels 
out  of  every  line  for  an  effective  frame  rate  of  7.5  trames/sce.  A  length  1 023  PN  sequence 
sent  at  a  20  megabit  rate  occupies  51.15  jus  leaving  1 2.3  ps  for  a  eoded  sync  pulse.  A 
cyclic  1023  bit  PN  sequence  ean  transmit  10  bits  of  information  via  cyclie  pulse  position 
modulation.  This  corresponds  to  27.1  dB  of  anti-jam  protection  as  measured  by  the  increased 
distance  between  code  v/ords  as  compared  to  a  direct  binary  transmission.  However  we  have 
only  transmitted  an  average  of  0.3  12  bits/pixel.  An  alternative  would  be  Lo  break  the  1023 
bit  sequence  into  two  sequences  of  length  5 1  1  each  lasting  25.6  ps.  Each  one  carries  9 
bits  of  information  for  a  total  of  18  bits  and  an  average  of  .563  bits/pixcl  at  24. 1  dB  anti¬ 
jam  protection.  This  is  further  illustrated  in  table  3.  As  ean  be  seen  Irom  tins  table  a 
sophisticated  coding  scheme  gives  approximately  a  5  dB  benefit  in  AJ.  Consider  the  same 
situation  for  the  100x100  pixel  charge  coupled  device  camera.  If  length  1023  PN  sequences 
are  clocked  at  20  megabits  per  second  twenty  51.15  microsecond  periods  could  be  used 
per  line  to  produce  a  frame  rate  of  about  10  per  second.  Performance  versus  codeword 
length  is  summarized  in  Table  4.  The  additional  anti-jam  benefit  from  die  coding  scheme 
gives  the  CID  camera  the  capability  of  running  at  a  higher  frame  rate  than  that  given  in 
Table  4  and  still  achieving  good  picture  quality  and  good  AJ  capability.  Table  5  summar¬ 
izes  the  performance  for  a  30  frame/sec  system.  Since  tins  system  is  capable  of  operating 
at  30  frames  per  second  with  good  AJ  and  good  picture  quality  (27.1  dB  AJ  at  1 .08  bits/ 
pixel),  it  offers  the  possibility  of  eliminating  a  potentially  costly  scan  converter  at  the 
ground  station.  The  picture  is  however,  either  of  lower  resolution  or  of  smaller  field  of 
view  than  the  vidicon  system. 

Table  3.  AJ  Improvement  for  Vidicon  Compatible  System  Using  PN-PPM 


Length  of 

No.  of  Sequences 

Decoded  Bits 

Total  No. 

Average  Bits 

AJ 

CR 

the  Sequence 

Transmitted 

per  Sequence 

of  Bits 

per  Pixel 

(dB) 

(dB) 

Table  4.  Performance  vs.  Codeword  Length  for  the  C1D 
System  at  10  Frames/sec  Using  PN-PPM. 


Length  of 
the  Sequence 

No.  of  Sequences 
Transmitted 

Decoded  Bits 
per  Sequence 

Total  No. 
of  Bits 

Average  Bits 
per  Pixel 

AJ 

(dB) 

CR 

(dB) 

1023 

20 

10 

200 

2.0 

27.1 

20.0 

511 

40 

9 

360 

3.6 

24.1 

17.4 

255 

80 

8 

640 

6.4 

20.1 

14.9 

Table  5.  Performance  vs.  Codeword  Length  for  the  CID 
System  at  30  Frames/sec  Using  PN-PPM. 


Length  of 
the  Sequence 

No.  of  Sequences 
Transmitted 

Decoded  Bits 
per  Sequence 

Total  No. 
of  Bits 

Average  Bits 
per  Pixel 

AJ 

(dB) 

CR 

(dB) 

1023 

6 

10 

60 

0.6 

27.1 

21.2 

51 1 

12 

9 

108 

1.08 

24.1 

17.9 

255 

24 

8 

192 

1.92 

21.1 

15.4 

DPCM  TRANSFORM  IMPLEMENTATION 

For  television  bandwidth  reduction,  it  has  been  shown  12,  3]  that  a  two-dimensional 
mixed  transform,  e.g.  Fourier  in  the  horizontal  direction  and  Hadamard  in  the  vertical  di¬ 
rection,  generally  gives  performance  intermediate  between  the  tv  o  transforms.  An  exception 
to  this  generalization  is  the  DPCM  transform.  The  combination  of  the  cosine  transform  in 
the  horizontal  direction  and  the  DPCM  transform  in  the  vertical  direction  has  performance 
equivalent  to  the  optimal  Karhunen  Loeve  transform,  as  shown  in  Appendix  A. 

This  result  has  been  demonstrated  both  theoretically  and  by  computer  simulation.  A 
hardware  DPCM  system  has  been  designed  and  is  being  constructed.  It.  is  meant  to  interface 
with  a  slow  scan  television  system  such  as  the  new  100  x  100  charge  coupled  device  cameras 
now  entering  the  market.  A  block  diagram  of  the  DPCM  system  is  shown  in  Figure  1. 

The  DPCM  has  been  designed  with  a  sixteen  level  nonlinear  quantizer  representing 
three  bits  of  magnitude  information  and  one  bit  of  sign  information.  The  quantization 


D2  D1  DqSIGN 
OUTPUTS 


Figure  1 .  The  DPCM  System. 

levels  can  be  set  to  arbitrary  levels.  Zero,  two,  three  or  four  bits  of  information  can  be 
transmitted  per  coefficient  depending  on  a  command  from  a  memory  unit.  The  memory 
unit  is  programmable  and  allows  the  operator  to  adjust  the  number  of  bits  per  coefficient 
to  the  coefficient  location.  The  system  is  also  designed  to  give  the  operator  control  of  the 
number  of  coefficients  sent.  The  analog  memory  is  being  simulated  by  an  A/D  converter, 
a  set  of  digital  shift  registers  and  a  D/A  converter.  An  appropriate  charge  coupled  device 
analog  memory  is  not  available  at  the  present  time.  From  the  experience  gained  in  the 
design  of  this  DPCM  encoder  it  appears  that  an  all  digital  DPCM  system  (except  for  the 
initial  summing  amplifier)  would  be  the  simplest  type  to  construct  and  has  significant 
power  and  weight  advantages  over  a  hybrid  encoder. 
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IMPLEMENTATION  OF  ONE-  AND  TWO-DIMENSIONAL 
DISCRETE  COSINE  TRANSFORMS 

Emphasis  has  been  placed  on  the  implementation  of  the  discrete  cosine  transform 
(DOT)  because  it  provides  a  very  close  approximation  to  the  optimum  Karhunen-Loeve 
transform,  while  permitting  compact,  real-time  implementation  by  a  combination  of 
multipliers  and  filters.  The  two  types  of  DCT  which  are  useful  for  reduced  redundancy 
television  image  transmission  are  obtained  by  extending  a  length  N  data  block  to  have 
even  symmetry,  taking  the  discrete  Fourier  transform  (DFT)  of  the  extended  data  block, 
and  saving  N  terms  of  the  DFT. 

The  “Odd  DCT”  (ODCT)  extends  the  length  N  data  block  to  length  2N-1  and  the 
“Even  DCT”  extends  the  length  N  data  block  to  length  2N.  For  example,  if  the  data 
block  were  ABC,  the  two  extensions  would  be  CBABC  and  CBAABC,  respectively.  Ana¬ 
lytic  expressions  for  these  transforms  are  given  in  Appendix  B. 

An  ODCT  or  EDCT  in  the  horizontal  direction  may  be  combined  with  an  ODCT  or 
EDCT  in  the  vertical  direction  to  give  three  different  types  of  two-dimensional  DCT. 

Three  different  ways  of  implementing  a  one-dimensional  DCT  in  real  time  have  been 
found,  requiring  filter  lengths  of  approximately  4N,  2N,  and  N  for  an  N-point  transform. 

A  detailed  comparison  is  provided  in  Table  1  of  Appendix  B. 

Two  of  the  three  types  of  two-dimensional  DCT  may  also  be  implemented  in  real¬ 
time  using  combinations  of  multipliers  and  fillers.  The  image  points  may  be  scanned  so 
as  to  produce  a  one-dimensional  sequence  whose  one-dimensional  transform  is  the  re¬ 
quired  two-dimensional  DCT  of  the  image  as  described  in  Appendix  B  and  illustrated  in 
Tables  2  and  3  of  that  appendix.  If  the  horizontal  and  vertical  sizes  of  the  extended  data 
block  have  no  common  divisor,  then  the  two-dimensional  DFT  required  for  a  two- 
dimensional  DCT  may  be  performed  as  a  one-dimensional  DFT  by  suitably  scanning  the 
extended  da+a  block  by  re-scanning  the  original  data  block. 

The  choice  of  a  hybrid  ODCT  by  even  DCT  permits  maximum  flexibility  in  the  block 
size,  since  any  square  block  size  may  be  used,  and  the  two-dimensional  transform  may  be 
implemented  using  the  switching  filter  EDCT,  shown  in  Figure  3  of  Appendix  B,  to  maxi¬ 
mize  the  block  size  attainable  with  a  limited  number  of  taps  per  filter.  In  this  case,  filters 


of  length  N2(2Nj-1  )  will  be  needed  for  a  bloek  size  of  N  j  by  Nt,  so  that  a  two-dimensional 
DCT  of  a  10  by  10  data  block  is  within  the  limits  of  present  bucket  brigade  device  filters. 

REPORT  SUMMARY 

This  report  describes  the  progress  on  the  third  phase  of  a  NUt  program  on  image 
bandwidth  reduction  for  application  to  the  ARPA  RPV  problem  of  sending  television 
images  over  spread  spectrum  channels.  This  report  details  the  second  quarter  of  a  hardware 
development  and  implementation  phase.  In  this  quarter  implementation  and  interfacing 
were  considered  for  the  two-dimensional  DCT  and  hybrid  IXT/Dl’CM  which  were  previ¬ 
ously  shown  to  provide  nearly  optimum  performance. 

Two-dimensional  discrete  cosine  transform  implementations  have  been  designed, 
but  their  utilization  awaits  development  of  a  compatible  sensor,  fwo-dimensional  hybrid 
DPCM/DCT  transform  hardware  is  being  built,  and  a  suitable  pseudonoise  pulse  position 
coding  format  has  been  shown  to  maximize  the  minimum  distance  between  code  words 
and  to  provide  a  significant  increase  in  AJ  protection  compared  to  simple  bit  stream  repe¬ 
tition.  The  improvement  is  about  5  dB  for  a  codeword  size  of  5  1  1  bits  with  a  compression 
ratio  of  78  to  1  and  is  a  slowly  varying  function  of  the  compression  ratio.  The  coding 
format  is  compatible  with  transmission  through  either  an  analog  or  a  binary  channel. 
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ABSTRACT 

Several  image  coding  systems  based  upon  transform  coding  have 
been  studied.  Analysis  and  simulation  has  shown  that  in  many  appli¬ 
cations  these  systems  are  capable  of  image  coding  at  about  one  to  two 
bits  per  pixel  with  acceptable  image  quality  and  tolerance  to  channel 
errors.  Implementation  studies  and  tests  that  have  been  performed 
indicate  that  the  systems  can  be  implemented  for  real  time  television 
operation  using  recently  developed  technologies. 

INTRODUCTION 

Transform  coding  techniques  have  been  explored  extensively  in 
theoretical  studies  and  by  simulation.  It  has  been  shown  that  a  signif¬ 
icant  bandwidth  reduction  can  be  achieved  in  many  applications  with 
minimal  image  degradation  and  relative  tolerance  to  channel  errors. 

The  major  drawback  of  transform  image  coding  for  real  time  television 
applications  in  the  past  has  been  computational  complexity.  However, 
recently  developed  technologies  such  as  acoustic  surface  wave  delay 
lines  and  charge  coupled  devices  have  made  transform  image  coding  at 
real  time  television  rates  feasible. 

CODING  TECHNIQUES 

Transform  Coding. 1 

In  transform  coding  systems  a  one  or  two  dimensional  mathe¬ 
matical  transform  of  an  image  line  segment  or  block  is  performed  at 
the  coder.  The  transform  coefficients  are  then  quantized  and  coded. 
At  the  receiver,  after  decoding,  an  inverse  transform  is  taken  to  obtain 
an  image  reconstruction.  Transforms  that  have  proven  useful  include 
the  Fourier,  Hadamard,  Slant,  Cosine,  and  Karhunen-Loeve  trans¬ 
forms.  A  bi'  rate  reduction  is  obtained  by  efficient  quantization  and 
coding  of  the  transform  coefficients.  Many  of  the  transform  coeffi¬ 
cients  of  a  natural  image  are  of  relatively  low  magnitude  and  can  be 
discarded  entirely,  or  coded  with  a  small  number  of  bits  per  coeffi¬ 
cient  while  maintaining  a  small  mean  square  error.  Simulation  studies 
indicate  that  a  bit  rate  reduction  to  about  1 .5  bits/pixel  can  be  ob¬ 
tained  for  monochrome  image  transform  coding  in  1 6  X  16  pixel 
blocks.  Color  images  require  about  2.0  bits/pixel.  Figure  I  shows  an 
original  and  encoded  pictures  for  two-dimen.-ional  transform  coding 
in  16  X  16  pixel  blocks  at  bit  rates  of  0.5  ano  I  bit  per  pixel. 

DPCM  Coding.2 

In  a  DPCM  system  the  value  of  a  scanned  image  sample  is  pre¬ 
dicted  and  the  difference  between  the  actual  and  the  predicted  value 
is  quantized  and  transmitted.  At  the  receiver  a  similar  predictor  uses 
transmitted  values  of  the  quantized  differential  signal  to  reconstruct  a 
replica  of  the  scanned  image.  Prediction  of  a  data  point  is  performed 
by  using  a  number  of  adjacent  previously  scanned  sample  values  where 
the  parameters  of  the  predictor  are  specified  in  terms  of  the  correla¬ 
tion  of  picture  elements. 

Properties  of  the  differential  signal  that  make  a  DPCM  system 
attractive  are  a  significant  reduction  in  the  variance  of  the  differential 
signal,  as  compared  to  the  variance  of  the  original  samples,  and  the 
fact  that  the  probability  density  function  of  the  differential  signal  is 
closely  approximated  by  an  exponential  function. 3  The  former 


(c)  I  Uit/Pixel 

Fig.  1.  Original  ami  cuonlcd  signal  using  two-dimensional  cosine 
transform  and  him  k  ifimntizatiim. 
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property  results  in  smaller  quantization  noise  power,  while  the  latter 
property  allows  for  designing  an  optimum  quantizer  to  obtain  a  fur¬ 
ther  reduction  of  noise  power.  Besides  these  two  improvements,  the 
quantized  differential  signal  has  a  smaller  entropy  than  the  quantized 
original  signal.  This  results  in  further  bandwidth  reduction  (cr  equiva¬ 
lently  improvements  in  signal  to  noise  ratio)  if  the  transmitted  signal 
is  entropy  coded.  With  standard  DPCM  systems,  monochromes  unages 
are  coded  with  2.5  to  3.0  bits/pixel. 

Comparison  of  Transform  and  DP  M  Coding.^ 

Both  DPCM  and  the  tran  form  coding  techniques  have  been  used 
with  some  success  in  coding  ..uctorial  data.  A  study  of  both  these  sys¬ 
tems  has  indicated  that  each  technique  has  some  attractive  character¬ 
istics  and  some  limitations.  Transform  coding  systems:  (a),  achieve 
superior  coding  performance  at  lower  bit  rates,  (b).  distribute  the  cod¬ 
ing  degradation  in  a  manner  less  objectionable  to  a  human  viewer; 

(c),  show  less  sensitivity  to  data  statistics  (picture-to-picture  variation); 
and  (d),  are  less  vulnerable  to  channel  noise,  compared  to  DPCM  cod¬ 
ing  systems.  On  the  other  hand,  DPCM  systems,  when  designed  to 
take  advantage  of  spatial  correlations  of  the  data:  (a),  achieve  a  better 
coding  performance  at  a  higher  bit  rate;  (b),  require  less  complex  cod¬ 
ing  equipment;  (c),  produce  a  minimal  coding  delay;  and  (d),  do  not 
require  a  large  block  memory,  as  compared  to  transform  coding 
systems. 

Hybrid  Transform/DPCM  Coding. 

A  serious  shortcoming  of  two-dimensional  transform  encoders  is 
the  requirement  of  a  block  memory.  To  avoid  this  problem  and  yet 
retain  the  attractive  features  of  transform  coding  systems,  a  hybrid 
encoder  that  uses  a  cascaded  unitary  transformation,  DPCM  encoder 
has  been  developed.  The  hybrid  encoder,  shown  on  Figure  2,  exploits 
the  correlation  of  the  data  in  the  horizontal  direction  by  taking  a 


one-dimensional  transform  of  each  line  of  the  picture,  then  it  operates 
on  each  column  of  the  transformed  data  us  ig  a  bank  of  DPCM  coders. 
The  DPCM  coders  replace  the  quantizers  1  a  two-dimensional  trans¬ 
form  coding  system.  Since  the  unitary  transformation  involved  is  a 
one-dimensional  transformation  of  individual  lines  of  the  pictorial 
data,  the  equipment  complexity  and  the  number  of  computational  op¬ 
erations  is  considerably  less  than  for  a  two-dimensional  transformation. 

The  hybrid  system  concept  can  be  extended  to  interframe  coding 
of  television  signals.  Such  a  coding  system  would  start  by  taking  a 
two-dimensional  transformation  of  each  block  of  the  television  signal, 
then  it  would  encode  the  transformed  signal  in  the  temporal  direction 
by  a  number  of  parallel  DPCM  encoders,  thus  exploiting  the  correla¬ 
tion  of  data  in  temporal  as  well  as  spatial  directions. 

Both  theoretical  and  the  experimental  results  indicate  that  a  hy¬ 
brid  system  employing  a  Karhunen-Loeve  transformation  prepuces  a 
better  result  than  a  hybrid  system  using  any  other  unitary  transforma¬ 
tion.  The  theoretical  results  for  Markov  data,  summarized  in  Figure  3, 
also  show  that  the  performance  of  the  Cosine  transform  system  is  only 
slightly  suboptimum  to  the  Karhunen-Loeve  transformation  system, 
and  is  superior  to  the  hybrid  Hadamard  or  Fourier  transform  systems. 
As  an  illustration  of  performance.  Figure  4  contains  encoded  pictures 
for  a  hybrid  transform/DPCM  coder. 

To  evaluate  the  effect  of  a  noisy  transmission  link,  hybrid  coded 
images  have  been  subjected  to  a  simulated  binary  symmetric  channel 
with  a  probability  of  bit  reversal  equal  to  p.  It  has  been  found  that  the 
inherent  error  propagation  in  the  DPCM  systems  of  the  hybrid  coder 
can  be  reduced  considerably  by  using  a  smaller  value  for  the  Aj's 
shown  in  Figure  2.  Naturally ,  the  best  value  for  the  Aj’s  depends  upon 
the  bit  error  probability  p.  For  p  =  I  O'  -  a  value  of  Aj  =  0.8  gives  near 
optimal  results.  Figure  4a  and  4b  show  encoded  pictures  for  a  hybrid 
coder  with  Aj  =  0.8  for  a  probability  of  error  of  p  =  10  -. 


Fig.  2.  Block  diagram  of  I  he  hybrid  system  using  a  cascade  of  one-dimensional  transformations 
and  a  bank  of  DPCM  systems  to  encode  pictorial  data 
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as  a  multiplication  of  the  vector  to  be  transformed  by  a  discrete  chirp, 
periodic  convolution  of  the  multiplied  vector  by  a  discrete  chirp,  and 
post  multiplication  by  a  discrete  chirp. 6 

Equation  (2)  may  be  implemented  using  real  multipliers  and  real 
filters.  This  configuration  is  shown  in  Figure  6.  With  the  further 
restriction  that  the  function  g  is  real,  non-negative  and  even,  G  is  real 
and  is  an  autocorrelation  function.  For  an  image  vector  of  size  N 
there  are  two  symmetrized  extensions  of  the  vector;  the  first  of  size 
2N,  the  other  of  size  2N-  1 .  Corresponding  to  the  DFT  of  these  two 
extended  vectors  there  are  two  discrete  cosine  transforms  (DCT) 
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Since  the  DFT  is  equivalent  to  the  Fourier  transform  of  a  periodic 
repetition  of  the  data  vector,  symmetrizing  the  input  signal  improves 
the  convergence  at  any  light-dark  edge  discontinuities  of  the  periodic- 
extension.  In  addition,  for  images  with  exponential  correlation  func¬ 
tions  Minted^  has  shown  that  the  l-'DCT  is  almost  indistinguishable 
from  the  Karliunen-Loeve  transform. 

However,  it  is  in  hardware  implementation  that  the  DCTs  excel 
since  they  may  be  computed  simply  as  the  real  part  of  a  DFT  via  the 
CZT  algorithm.  The  CZT  algorithm  is  a  "real-time  ”  algorithm  which 
makes  it  possible  to  compute,  in  natural  frequency  order,  Fourier  co¬ 
efficients  at  the  same  rate  as  the  data  is  being  gathered.  In  addition, 
the  computation  involves  only  multiplication  and  convolution  which 
can  be  performed  by  analog  multipliers  and  linear  filters  respectively. 
Thus  analog  to  digital  conversion  of  the  video  is  not  required  nor  is 
there  a  need  for  special  purpose  digital  computations. 

Two  technologies  are  available  for  the  implementation  of  the 
convolution  filters. surface  acoustic  waves  (SAW)  and  charge  transfer 
devices  (CTD).  The  SAW  technology  uses  acoustic  waves  on  piezo¬ 
electric  crystals  with  deposited  electrodes  while  the  CTD  uses  an  elec¬ 
tric  tield  to  induce  propagation  of  concentrations  of  minority  earriers 
at  a  silicon-silicon  oxide  boundary  in  a  monolithic  silicon  analog  inte¬ 
grated  circuit.  In  both  technologies  the  required  linear  filters  are 


implemented  as  transversal  filters  whose  impulse  responses  are  deter¬ 
mined  by  the  metalization  pattern  deposited  during  fabrication. 

file  transversal  filter  was  first  proposed  by  Kallmann^  and  con¬ 
sists  of  a  non-dispersive  delay  line  contiguously  tapped  along  its  length 
transverse  to  the  direction  of  signal  propagation  with  lightly  coupled 
non-interacting  taps.  The  impulse  response  of  this  filter  is  uniquely 
speed ied  by  the  amplitude  and  polarity  of  the  taps  and  any  bounded, 
linite  duration  impulse  response  of  specified  bandwidth  can  be  synthe¬ 
sized  by  sampling  the  desired  response  signal  at  its  Nyquist  rate  and 
setting  the  tap  weights  along  the  delay  line  to  these  values.  Although  a 
transversal  lilter  normally  implements  a  non-periodie  convolution, 
periodic  convolution  can  he  achieved  by  a  number  of  techniques;  (1) 
the  input  signal  can  be  repeated;  (2)  the  output  of  the  delay  line  can 
be  recirculated  to  the  input;  (3)  tile  impulse  response  can  be  made  two 
periods  of  the  desired  function;  or  (4)  an  auxiliary  delay  line  can  be 
employed. 

In  order  to  continuously  transform  the  input  signal,  this  signal 
must  he  subdivided  into  blocks  whose  lengths  are  the  same  size  as  the 
size  ol  the  appropriate  Fourier  transform  and  two  sets  of  convolution 
tillers  used  alternately.  However,  the  pre-  and  post-multipliers  may  be 
multiplexed  between  the  two  groups  of  filters.  By  these  techniques  a 
logvN  increase  in  throughput  is  possible  over  a  corresponding  fast 
Fourier  translorm  (I  TT)  operating  at  the  same  sample  rate  and  em¬ 
ploying  one  complex  multiplier.  This  results  in  a  reduction  of  size  and 
power  since  no  quantization  is  required  and  calculation  proceeds  at  the 
data  sample  rate. 

When  a  vertical  rlifferential  pulse  code  modulator  (DPCM)  modem 
follows  a  horizontal  cosine  transform  a  very  compact  digital  real-time 
image  transmission  system  results.  As  shown  in  part  2,  this  system 
achieves  performance  essentially  indistinguishable  from  that  of  the 
Karhunen-Loeve  transform  for  pictures  witlt  exponential  correlation 
functions  without  having  to  employ  two  dimensional  transform  encod¬ 
ing.  However,  il  two  dimensional  Fourier  encoding  is  required  it  may 
be  achieved  by  concatenation  of  two  one  dimensional  transforms  with 
auxiliary  memory  to  transpose  the  partial  transform  from  row  to  col¬ 
umn  format;  or  it  may  be  achieved  by  linear  congrucntial  scanning  of 
the  two  dimensional  area  and  the  use  of  a  one  dimensional  transfor¬ 
mation  as  described  under  two  dimensional  transforms. 


Fig.  6.  DFT  via  CZT  algorithm  with  parallel  implementation  of  complex  arithmetic. 

A-6 


Two  Dimensional  Transforms 


The  alternative  expressions  for  the  ODCT  and  EDCT  shown  in 
eqs.  (5)  and  (6)  make  evident  their  implementation.  The  appropriate 
premultiplier  reference  function,  transversal  filter  tap  weights  and 
postmultiplier  reference  function  for  the  ODCT  and  EDCT  are  given 
by  a,  b,  c  of  eqs.  (7)  and  (8)  respectively. 


where  sq  =  .5  and  Sjj,  =  1  for  j  >  0 


fp  =  g  (J  1 , )2> 

(12) 

q(k|,k2)  =  k|V|M2  +  k2v2M| 

(MOD  M) 

(13) 

Fq  =  G (k|, k2) 

(14) 

M2UjV|  =  1  (MODMj) 

(15) 

^1U2V2  =  1  (MOD  M->) 

(16) 

The  incorporation  of  a  two-dimensional  DPT  device  using  the 
linear  congruential  scan  into  a  reduced  redundancy  image  transmission 
system  is  shown  in  Figure  7.  Since  unequal  quantization  would  nor¬ 
mally  be  applied  to  the  transform  output  values,  the  output  scan  coor¬ 
dinates  available  at  B  and  C  may  be  used  to  keep  track  of  the  coordi¬ 
nates  of  the  output  value  at  A. 
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For  the  EDCT 
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A  two-dimensional  DFT  ol  an  M  |  by  M2  data  block  may  be  per¬ 
formed  as  a  one-dimensional  D1 T  of  size  M  =  M  |  M2  of  a  sequence 
obtained  by  an  appropriate  linear  congruential  scan  of  the  data  block 
provided  that  M[  and  M2  have  no  common  divisor.8>l° 


The  equivalent  two  and  one  dimensional  DFTs  are  given  in  eqs. 

(9)  and  (10)  respectively,  with  the  input  scan  defined  by  eqs.  (11)  and 
( 4  2),  *1 J  Hr  •.  jWptH  TefhrcJ  by  eqs.  The  constants 

U1 ,  vi  •  U2,  ”2  may  be  any  solutions  of  eqs.  (15)  and  (16). 

M ,  - 1  M2- 1 

G(ki,k2)  =  22  g(il’j2> 

j 1 =0  J2=0 


(9) 


for  q  =  0, .  .  .  M-  1  (10) 

P  (j  1 , J2>  =  j  1  u1m2  +J2u2M1  (MOD  M)  (11) 


A  two-dimensional  DCT  of  an  Nj  by  N2  data  block  may  be 
regarded  as  a  two-dimensional  DFT  of  an  M  |  by  M2  data  block,  where 
<Mk  =  2N|<  or  M|(  =  2N|<-  1  depending  on  whether  an  EDCT  or  an 
ODCT  is  used  in  the  kth  direction.  If  M  |  and  M2  have  no  common 
divisor,  the  two-dimensional  DCT  may  therefore  be  performed  as  a 
one-dimensional  DFT  using  CZT  hardware.  The  configuration  of 
Figure  8  is  therefore  appropriate  for  the  two  dimensional  DCT  as  well 
as  the  two-dimensional  DFT.  For  the  two-dimensional  DCT,  however, 
the  extended  data  block  is  scanned  by  an  appropriate  rescan  of  the 
original  data  block,  and  a  modified  scan  generator  will  be  needed.  In 
addition,  the  one-dimensional  DFT  size  needed  to  perform  a  two- 
dimensional  DCT  in  this  way  is  M  iM2,  which  is  almost  four  times 
larger  than  N|N2.  If  the  symmetry  of  the  one-dimensional  sequence 
generated  by  the  linear  congruential  scan  of  the  extended  data  block 
is  exploited,  the  length  M  ]M2  one-dimensional  DFT  may  be  replaced 
by  a  one-dimensional  DCT  of  about  half  that  length,  thus  permitting  a 
substantial  reduction  in  the  number  of  taps  needed  in  the  filters. 

Transform  Hardware. 

This  section  will  describe  how  both  surface  acoustic  wave  devices 
and  charge  transfer  devices  have  been  used  to  implement  the  chirp-Z 
algorithm  for  bandwidth  reduction.9’10  The  surface  acoustic  wave 
system  was  constructed  to  be  compatible  with  standard  television.  The 
charge  transfer  device  system  was  constructed  to  be  compatible  with  a 
slow  scan  television  camera  such  as  the  new  100  X  100  charge  coupled 
device  cameras  now  on  the  market.  A  second  charge  transfer  device 
system  is  being  constructed  to  be  compatible  with  standard  television. 

A  block  size  of  32  has  been  chosen  for  a  feasibility  demonstration 
of  a  surface  acoustic  wave  implementation  of  die  real-time  discrete 
Fourier  transform.  With  a  conventional  line  scan  time  of  53.5  micro¬ 
seconds  and  a  sample  rate  of  256  samples  per  line,  a  4.78  MHz  sam¬ 
pling  rate  is  required.  A  block  of  32  points  transforms  6.69  microsec¬ 
onds  or  one  eighth  of  the  video  signal.  The  total  number  of  taps  on 
the  filter  is  63,  i.e. ,  2N- 1 .  Two  SAW  devices  have  been  constructed  at 
the  Naval  Undersea  Center.  One  is  used  to  generate  the  32  point  sine 
and  cosine  chir  synals  for  i*-e-  and  postmultifjicattou  TU  ttuv  it 
used  to  convolve  the  multiplied  input  with  the  four  transversal  filters 
as  shown  in  Figure  6.  A  photograph  of  the  SAW  device  in  shown  in 
Figure  8. 

The  feasibility  of  the  charge  transfer  device  implementation  has 
been  demonstrated  with  a  bucket  brigade  transversal  filter.  The  bucket 
brigade  devices  have  200  taps  and  can  operate  at  a  sampling  rate  of 
several  hundred  kilohertz.  This  makes  them  ideal  for  slow  scan,  low 
resolution  TV  such  as  the  recent  100  X  100  charge  coupled  device 
cameras.  Tile  tap  weights  of  the  bucket  brigade  device  had  been  cho¬ 
sen  for  another  purpose.  The  devices  were  built  by  Texas  Instruments 
under  Rome  Air  Development  Center  contract  #  F30602-73-C-0027 
for  an  application  other  than  bandwidth  reduction.  Fortunately  the 
tap  weights  had  been  chosen  so  as  to  approximately  implement  a 
cosine  transform.  Devices  are  being  redesigned  with  the  tap  weights 

necessan  for  an  exart  nrtrl  rw'ne  irnnil  yni  TIb’h(v:'  liaiTl.’i  li  f 

transform  system  has  been  constructed.  The  exact  cosine  transform 
system  would  have  given  a  single  pulse  width  of  10  microseconds  for  a 
line  scan  time  of  one  millisecond  and  a  sample  rate  of  100  samples  per 
line  The  approximate  cosine  system  gives  an  output  that  is  function¬ 
ally  like  sin  x/x.  The  width  and  shape  of  the  output  agree  with 
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Fig.  7.  Reduced  redundancy  image  transmission  system  using  two-dimensional  discrete 
Fourier  transform  device  with  linear  cungruential  scan. 


CZT  TRANSVERSAL  FILTER 

Fig.  8.  S4  W  device. 


computer  simulation  of  the  approximate  cosine  transform.  This  im¬ 
plies  the  filtfrs  ami  the  multipliers  are  performing  adequately. 

Both  the  charge  transfer  device  system  and  the  surface  acoustic- 
wave  system  are  light  weight,  low  power,  low  cost  systems  that  pro¬ 
vide  a  capability  of  real-time  bandwidth  reduction. 

SUMMARY 

Several  image  transform  coding  systems  have  been  described, 
analyzed,  and  tested  by  simulation.  It  has  been  shown  that  the  two 
dimensional  Cosine  transform  system  and  the  hybrid  Cosine  transform 
DPCM  system  both  provide  significant  bandwidth  reduction  with 
acceptable  image  quality  and  tolerance  to  channel  errors.  Implemen¬ 
tation  techniques  particularly  suited  to  these  coding  systems  have  been 


developed  using  acoustic  surface  wave  delay  lines  and  charge  coupled 
devices  for  coding  at  real  time  television  rates.  Also,  post  processing 
techniques  have  been  demonstrated  for  the  reduction  of  quantization 
error  in  transform  components. 
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APPENDIX  B 

HIGH  SPEED  SERIAL  ACCESS  IMPLEMENTATION 
FOR  DISCRETE  COSINE  TRANSFORMS 

Jeffrey  Speiser 
Naval  Undersea  Center 


INTRODUCTION 


Two  different  types  of  discrete  cosine  transform  (DCT)  are  useful  for 
reduced  redundancy  television  image  transmission  [1-3].  Both  are  obtained  by 
extending  the  length  N  data  block  to  have  even  symmetry,  taking  the  discrete 
Fourier  transform  (DFT)  of  the  extended  data  block,  and  saving  N  terms  of  the 
resulting  DFT.  Since  the  DFT  of  a  real  even  sequence  is  a  real  even  sequence, 
either  DCT  is  its  own  inverse  if  a  normalized  DFT  is  used. 

The  "Odd  CDT"  (ODCT)  extends  the  length  N  data  block  to  length  2N-1,  with 
the  middle  point  of  the  extended  block  as  a  center  of  even  symmetry.  The 
"Even  DCT"  (EDCT)  extends  the  length  N  data  block  to  length  2N,  with  a  center 
of  even  symmetry  located  between  the  two  points  nearest  the  middle.  For 
example,  the  odd  length  extension  of  the  sequence  ABCisCBABC,  and  the 
even  length  is  C  B  A  A  B  C.  In  both  cases,  the  symmetrization  eliminates  the 
jumps  in  the  periodic  extension  of  the  data  block  which  would  occur  if  one 
edge  of  the  data  block  had  a  high  value  and  the  other  edge  had  a  low  value; 
in  effect  it  performs  a  sort  of  smoothing  operation  with  no  loss  of  information. 

It  will  be  noted  that  the  terms  "odd"  and  "even"  in  ODCT  and  EDCT  refer  only  to 
the  length  of  the  extended  data  block  -  in  both  cases  the  extended  data  block 
has  even  symmetry. 

Both  types  of  DCT  may  be  implemented  using  compact,  high  speed,  serial 
access  hardware,  in  structures  similar  to  those  previously  described  [1,2]  for 
the  Chirp- Z  transform  (CZT)  implementation  of  the  DFT.  The  significant  difference 
between  the  ODCT  and  the  EDCT  is  the  block  size  of  the  transform  which  may  be 
implemented  using  transversal  filters  having  a  given  limit  on  the  number  of 
taps.  A  comparison  of  different  high  speed  implementations  of  the  DCT  for  a 
block  size  of  N  is  given  in  Table  1. 
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ODD  DISCRETE  COSINE  TRANSFORM 


EVEN  DISCRETE  COSINE  TRANSFORM 


I } 

a  l 


The  EDCT  of  g  is  defined  by  equation  (7) ,  where  the  extended  sequence  is 
defined  by  equation  (8) . 


Gek  =  e 


g-l-j  gj 


-  ink  N-l  -  i2njk 
2N  _  2N 


2  *j 

j-  -N 


for  k  =  0,  1,  .  .  . ,  N-l 


for  j  =  0,  1,  .  .  N-l 


(7) 


(8) 


If  the  mutually  complex  conjugate  terms  in  equation  (7)  are  combined, 
then  equation  (9)  results.  Equation  (9)  may  be  viewed  as  an  alternate  way  of 
defining  the  EDCT. 

-  ink  N-l 

,  =  9  Do  I  , 

ek 


-  i2njk 

r  -  9  j  a  2N  Q  2N  v  _  9 

G.i.  -  2  Re  \  e  ^  gj  e  {  2 

j=0  )  J=0 


S, 


cos 


2n(j  +  .5)k 


2N 


Equation  (9)  may  be  put  in  either  of  the  CZT  formats  given  in  equations 
(10)  and  (11),  where  gj  is  defined  by  equation  (12). 
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gj,  j  =  0,  1,  .  .  .,  N-l( 
0,  j  =  N,  .  .  .,  2N-1 


(12) 
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IMPLEMENTATION  OF  THE  DCT  VIA  THE  CZT 


A  general  DFT  of  length  M,  defined  by  equation  (13)  may  be  computed  by 
CZT  defined  by  equations  (15)  and  (16)  as  shown  in  Figure  1. 


Figure  1  -  General  DFT  Implemented  via  a  CZT 
Using  a  Transversal  Filter. 


It  will  be  noted  that  the  postmultiplier  of  Figure  1  is  ready  to  produce 
the  first  transform  point  when  the  first  input  to  the  filter  is  lined  up  with 
the  central  tap. 

If  the  ODCT  is  viewed  as  a  DFT  using  equation  (1),  then  it  may  be  imple¬ 
mented  using  the  structure  shown  in  Figure  1  with  M  =  2N-1,  and  the  required 
filter  length  is  2M-1  which  equals  4N-3. 

The  EDCT  as  defined  by  equation  (7)  may  be  implemented  similarly  with 

2 

M  =  2N  by  changing  the  postmultiplier  weights  to 

e  2N 

R.  Means  has  noted  that  a  twofold  reduction  in  the  required  length  of 
the  filter  and  read  only  memories  is  possible  [4]  when  the  ODCT  is  computed  via 
equation  (3)  with  a  CZT  used  to  perform  the  required  DFT,  Since  only  N  terms 
of  the  input  are  non- zero,  and  only  N  terms  of  the  transversal  filter's  output 
are  needed,  only  the  first  N  outputs  of  the  ROMs  and  the  central  2N-1  taps  of 
the  transversal  filter  of  Figure  1  are  needed  when  the  ODCT  is  computed  this 
way.  A  similar  conclusion  holds  for  the  EDCT  computed  via  equation  (9)  with 
a  CZT  used  to  compute  the  required  DFT. 
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IMPLEMENTATION  OF  THE  EDCT  USING  SNITCHING  FILTERS 


The  computationally  difficult  step  in  the  realization  of  the  EDCT  via 
equation  (10)  is  the  convolution-like  operation  shown  in  equation  (16). 

.2 

for  k  =  0,  1,  .  .  • ,  N-l 


A. 


N-l 

k  ■  E  ai e 
j-o 


iir(j-k)1 
2N 


If  P  is  defined  by  equation  (15)  with  M  —  2N,  the  convolution  like 
character  of  equation  (16)  may  be  made  more  explicit  as  shown  in  equation  (17), 


N-l 


N-l 


S  aj  Pj-k  =  S  aj  Pk-j 

j=0  j=0 


Equation  (17)  would  represent  a  (periodic)  convolution  or  correlation  if 
the  P  sequence  had  period  N.  In  our  case,  however,  the  P  sequence  has  period 
2N.  The  essential  symmetries  of  the  P  sequence  are  given  by  equations  (18) 
and  (19). 


P_  =  P 


-s 


N-s 


=  e 


iTT(N2-2Ns+s2) 
2N 


=  (-D  e 


iirN 

2 


For  specific  values  of  N,  equation  (19)  can  be  simplified  further  by 
using  equation  (20) . 


iirN 


1,  N  =  0  (mod  4) 
i,  N  =  1  (mod  4)' 
|-1,  N  =  2  (mod  4) 
■i,  N  =  3  (mod  4)  ) 
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(16) 


(17) 


(18) 


(19) 


(20) 


■ . 


Table  2  shows  the  weights  needed  to  evaluate  (17),  where  identity  (18) 
has  been  used  to  eliminate  negative  indices.  Table  3  shows  the  same  weights 
after  additional  simplification  using  identities  (19)  and  (20)  when  N  is  a 
multiple  of  four, to  view  the  weights  needed  at  successive  times  as  modifications 
of  cyclic  shifts  of  Pq,  P^,  .  .  .,  P^_^.  Table  4  shows  the  weights  of  Table  3 
arranged  to  correspond  to  cyclic  shifts  of  the  data,  corresponding  to  an 
implementation  using  length  N  transversal  filters  with  data  points  read  twice. 

SHIFT 


Table  2  -  Weights  Required  for  the  Correlation- Like  Operation 
in  the  TDCT,  Ordered  for  Fixed  Data  Position. 
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SHIFT 

INDEX 


WEIGHT  SET 


k  PN-k-l '  P1  P0 . 


(-1)k'Vk+i  c-u^N-k 


Table  3  -  Reduced  Form  of  the  Weights  Required  for  the  Correlation-Like  Operation 
in  the  EDCT  when  N  =  0  (Mod  4),  Ordered  for  Fixed  Data  Position. 


SHIFT 

INDEX 
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Vi  • 

•  • 

1 

"PN-1 

P 

N-2 

2 

-PN-1 

PN-2 

3 

"PN-1 

PN-2 

WEIGHT  SET 


Table  4  Weights  Required  for  the  Correlation-Like  Operation  in  the  EDCT  when 
N  =  0  (Mod  4)  Ordered  for  Cyclically  Shifting  Data. 
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The  required  time-variant  operations  may  be  moved  out  of  the  filters  as  shown 
in  Figure  2. 


Figure  2  -  Switching  Filter  for  use  in  the  EDCT  for 
N  =  0(Mod  4)  or  N  =  2(Mod  4) 

A  complete  EDCT  structure  using  the  switching  filter  is  shown  in  Figure  3. 
The  switch  changes  position  with  every  data  shift,  and  its  initial  position  is 
determined  by  whether  the  block  size  is  congruent  to  zero  or  two  modulo  four. 

A  similar  structure  may  be  used  for  block  sizes  congruent  to  one  or  three 
modulo  four,  except  that  ROM  1  and  ROM  2  will  contain  ±  i  entries  instead  of 


±  Vs.  Because  o£  the  simple  structure  of  the  output  needed  from  ROM  1  and 
ROM  2  it  may  be  preferable  to  replace  them  with  a  combination  of  a  counter  and 
gates  to  generate  the  required  functions. 


ROM, 


Figure  3  -  EDCT  Architecture  using  Switching  Filter 


CONCLUSIONS 


The  three  general  types  of  DCT  implementations  using  transversal  filters 
and  multipliers  each  possess  unique  advantages.  A  DCT  implementation  using  a 
complete  CZT  will  use  transversal  filters  with  tap  weightings  which  should 
become  standard  components  in  the  near  future.  An  implementation  using 
shortened  versions  of  the  filters  and  ROMs  needed  for  a  standard  CZT  permits  a 
longer  block  length  transform  to  be  implemented  for  a  given  number  of  taps  per 
filter,  and  simultaneously  eliminates  the  need  to  rescan  data  points.  A  new 
structure  called  the  "switching  filter  EDCT"  permits  a  further  increase  in 
block  size  for  a  given  filter  length,  but  requires  more  filters,  some  additional 
switching  circuitry,  and  a  rescan  of  the  data.  The  switching  filter  implemen¬ 
tation  of  the  EDCT  uses  transversal  filters  each  having  a  number  of  taps  equal 
to  the  transform  block  length.  The  other  implementations  require  nearly  twice 
as  many  and  four  times  as  many  taps  per  filter. 
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INTRODUCTION 


Two  different  types  of  one- dimensional  discrete  cosine  transform  (DCT) 
called  the  Even  DCT  (EDCT)  and  Odd  DCT  (ODCT)  which  are  useful  for  reduced 
redundancy  image  transmission  [1-3]  may  be  combined  to  form  three  distinct 
types  of  two-dimensional  discrete  cosine  transform.  Each  of  the  two-dimensional 
DCTs  may  be  defined  as  a  two-dimensional  discrete  Fourier  transform  (DFT)  of  a 
doubly  symmetrized  extension  of  the  data  block  in  which  the  extended  data 
block  has  even  symmetry  about  each  of  two  axes.  If  the  original  data  block 
size  was  by  N2,  the  extended  block  size  may  be  2N^  by  21L  for  the  EDCT  by 
EDCT,  2NX-1  by  2N2  for  the  ODCT  by  EDCT,  or  2^-1  by  2N2-1  for  the  ODCT  by  ODCT. 

Although  the  three  types  of  two-dimensional  DCT  may  be  expected  to  provide 
similar  performance  for  television  image  redundancy  reduction,  they  differ 
markedly  with  respect  to  difficulty  of  real-time  implementation.  Each  of  the 
two  transforms  may,  of  course,  be  implemented  by  performing  a  line-by-line  one¬ 
dimensional  partial  transform,  using  an  auxiliary  "comer- turning"  memory  to 
store  and  transpose  the  partial  transform,  and  then  perform  a  partial  transform 
in  the  second  dimension. 

It  has  previously  been  shown  that  a  two-dimensional  DFT  may  be  implemented 
as  a  one- dimensional  DFT  of  size  M  =  MjM2  of  an  appropriate  linear  congruential 
scan  of  an  by  M2  data  block,  provided  that  and  M2  have  no  common  divisor 
[1].  Since  2N^  and  2N2  have  the  common  divisor  of  2,  the  linear  congruential 
scan  does  not  aid  in  performing  a  two-dimensional  EDCT.  For  a  two-dimensional 
ODCT,  the  linear  congruential  scan  will  be  compatible  with  certain  block  sizes, 
but  never  when  the  block  is  square.  For  a  hybrid  ODCT  by  EDCT  two-dimensional 
transform,  the  linear  congruential  scan  is  compatible  with  all  square  block 
sizes,  since  2N  and  2N-1  never  have  a  common  divisor. 
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It  will  be  shown  that  the  linear  congruential  scan  converts  a  hybrid  ODCT 
by  EDCT  of  an  N  by  N  data  block  into  a  length  N(2N-1)  one -dimensional  EDCT  which 
may  be  implemented  via  a  switching  filter  EDCT  [4]  using  transversal  filters  of 
length  N(2N-1).  For  a  given  upper  limit  on  the  number  of  taps  in  the  transversal 
filter,  the  hybrid  ODCT  by  EDCT  therefore  permits  a  larger  block  size  than  would 
be  possible  with  a  two-dimensional  ODCT,  and  a  vastly  simpler  implementation  than 
would  be  needed  for  a  two-dimensional  EDCT.  A  comparison  of  the  two-dimensional 
DCT  implementations  for  an  by  N2  block  size  is  given  in  Table  1. 
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SYMMETRIES  PRESERVED  BY  SELECTED  TOROIDAL  SCANS 


For  a  two-dijnensional  DFT  to  be  equivalent  to  a  one-dimensional  DFT, 
the  two-dimensional  block  size  ^  by  M2  nust  be  such  that  and  M,  have  no 

divisor.  In  that  case,  the  two-dimensional  DPT  of  equation  (1)  will  be 
equivalent  to  the  one-dimensional  DFT  of  equation  (2)  with  M  =  M^.  The 
appropriate  linear  congruential  input  scan  is  defined  by  equations  (3)  and  (4), 
and  the  associated  output  scan  is  defined  by  equations  (5)  and  (6) .  The 
constants  up  vp  up  v2  may  be  any  solution  of  equations  (7)  and  (8). 
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k2  "  0,  .  .  . ,  M2-I 


p=0 


for  q  =  0,  .  .  m-i 


p(jl»j2)  -  J'iuiM2  +  J2u2Mi  (K>D  M) 


fp 


qC^i  ,k2D  -  k-jV-jR,  +  k2v2M1 
Fq  "  Gfki»k2) 


(M3D  M) 


M2U1V1  "  1 


(M3D  M, ) 


M1U2V2  =  1  (MOD  M,) 
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(8) 
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The  behavior  of  the  input  scan  is  extremely  simple  to  describe  if  v^  and 
v2  are  both  chosen  equal  to  one.  In  this  case,  to  move  from  the  presently 
scanned  point  to  the  next  point,  each  coordinate  is  incremented  by  1,  and  the 
first  coordinate  is  reduced  modulo  and  the  second  coordinate  is  reduced 

modulo  M2,  as  shown  in  equation  (9).  The  notation  (a)^  denotes  the  residue  of 
a  modulo  b.  Such  a  scan  may  be  viewed  as  a  spiral  on  the  surface  of  a  torus. 

The  required  scan  generator  is  simply  a  pair  of  counters  and  digital  to  analog 
converters . 

p((s+DMi>  (s+1)m2)  =  p(^Mi’(s:)M2)  +  1  (Modulo  M)  (9) 

Equation  (9)  will  now  be  proved.  From  (7)  and  (8)  it  follows  that 
(M2u^-1)  is  a  multiple  of  and  (M^-l)  is  a  multiple  of  N^.  The  product 
is  therefore  a  multiple  of  MjM2  as  shown  in  equation  (10) . 

(M^-l)  (M^-l)  =  0  (Modulo  M)  (10) 

Sinplification  of  (10)  yields  (11) . 

p(l,l)  =  M2u1  +  MjU2  =  1  (Modulo  M)  (11) 

Equation  (12)  follows  from  the  scan  defining  equation  (3). 

p((s+1)M1,(s+i:)M2)=  (s+1)uiM2  +  (s+1)u2M1  =  P^^’^M,,)^1’1)  (i2) 

The  desired  equation  (9)  follows  from  (11)  and  (12).  Since  the  scan 
takes  the  two-dimensional  origin  (0,0)  into  the  one -dimensional  origin  0, 
equation  (9)  may  be  solved  by  induction  to  yield  the  closed  form  scan  equation 
(13). 

K(sV  ts)M2) ’ s 
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(Modulo  M) 


(13) 


Equation  (13)  says  that  the  kth  coordinate  of  the  point  scanned  at  time  s  is 
obtained  by  reducing  s  modulo  M^. 

The  scanning  of  a  data  block  which  has  been  symmetrized  for  the  two- 
dimensional  ODCT  will  now  be  considered  in  detail.  The  data  block  is  extended 
using  the  double  mirror  symmetry  defined  in  equation  (14) . 

gt^l*  ±j 2 )  =  gCjx»  j2) 


The  two-dimensional  DFT  of  the  extended  data  block  defines  the  two- 


dimensional  DCT  of  the  original  data  block  given  in  equation  (15),  with 
Mk  -  2Nk-l. 


G(-kl,k2^  "  S  S  g(j1»j2)  e 
h=  -^r1)  j2=  '(n2_1) 


Since  the  indices  are  only  defined  modulo  and  M2  respectively,  the 
summation  limits  in  (15)  are  really  no  different  from  those  in  (1).  If  the 
scan  is  defined  by  equations  (3)  and  (4) ,  then  the  symmetry  of  the  corresponding 
one- dimensional  sequence  is  shown  in  equation  (16). 


f.p  -  gC-jj.-jp  -  gOpip  -  fp 

The  toroidal  scan  of  the  extended  data  block  -  which  is,  of  course, 
obtained  by  repeatedly  scanning  points  of  the  original  data  block  -  is 
illustrated  in  Table  2  for  a  block  size  of  2  by  3.  The  numbers  in  the  table 
indicate  the  scan  order,  while  the  letters  indicate  the  data  values. 


Table  2A  -  Toroidal  Scan  for  an  ODCT  of  a  2  by  3  Data  Block 


Table  2B  -  One- Dimensional  Sequence  Produced  by  a  Toroidal  Scan 
for  an  ODCT  of  a  2  by  3  Data  Block. 


INDEX 


DATA  VALUE 


The  symmetry  of  a  sequence  produced  by  a  toroidal  scan  of  a  data  block 
symmetrized  for  a  hybrid  ODCT  by  EDCT  will  now  be  examined.  The  symmetrization 
for  the  hybrid  two-dimensional  ODCT  by  EDCT  is  described  by  the  requirement 
that  each  pair  of  points  of  the  extended  data  block,  (jjj2)  ^  (jpj2)  ^ave 
the  same  data  value  whenever  the  points  satisfy  equation  (17). 


*  * 


jx  =  51  or  jx  +  jx  =  0 

(Modulo  M^) 

(17A) 

^  ft 

h  =  h  or  h  *  h  =  -1 

(Moaulo  M2) 

(17B) 

Equations  (18)  and  (19)  examine  the  point  scanned  k  samples  after 
(NpN2)  and  k  samples  before  the  previous  point  (N^-l.N^-l). 

P<W  +  k  ’  P((Nl*k\-  tN2+«M2) 

(Modulo  M) 

(18) 

pfNj-l.^-l)  -  k  -  p^-l-k)^,  (N2-l-k)M_ 

^  (Modulo  M) 

(19) 

The  sum  of  the  first  coordinates  for  the  points 

of  equations  (18)  and  (19)  is  given  in  equation  (20) . 

for  the  second  coordinates  is  given  in  equation  (21) . 

on  the  right-hand  side 

The  corresponding  sum 

(Nx+k)  +  (Nx-l-k)  =  2NX-1  =  0 

(Modulo  M^) 

(20) 

(N2+k)  +  (N2-l-k)  =  2N2-1  =  -1 

(Modulo  M2) 

(21) 

It  therefore  follows  from  equation  (17)  that  the  two  points  have  the 

same  data  value  as  shown  in  equation  (22) . 

f(p(N1,N2)  +  k)  =  ^(^-1,^-1)  -  k) 

(22) 

! 


If  the  toroidal  scan  is  therefore  used,  starting  at  point  (NpN^J  of  the 
extended  image,  the  resulting  sequence  will  have  the  symmetry  required  for  a 
one- dimensional  EDCT.  The  toroidal  scan  of  a  2  by  2  data  block  extended  for 
the  two-dimensional  hybrid  ODCT  by  EDCT  is  shown  in  Table  3. 


Table  34  -  Toroidal  Scan  of  a  2  by  2  Data  Block  Examined  for  the 
Two-Dimensional  Hybrid  ODCT  by  EDCT. 


D  C  C  D 
2  11  8  5 


B 

6 

D 

10 


A 

3 

C 

7 


A 

0 

C 

4 


B 

9 

D 

1  ! 


Table  3B  -  One-Dimensional  Sequence  Produced  by  the 
Toroidal  Scan  of  a  2  by  2  Data  Block  Symmetrized 
for  a  Two-Dimensional  Hybrid  ODCT  by  EDCT. 


INDEX 

8 

9 

10 

11 

0 

1 

2 

3 

4 

5 

6 
7 


DATA  VALUE 


C 

B 

D 

C 

A 

D 

D 

A 

C 

D 

B 

C 
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LINEAR  CONGRUENTIAL  SCAN  GENERATORS  FOR  SYMMETRIZED  DATA  BLOCKS 


A  toroidal  scan  of  an  actual  Mj  by  R,  data  block  was  shown  by  equation 
(9)  to  be  obtained  by  successively  adding  1  to  each  of  the  scan  coordinates 
and  reducing  the  sums  modulo  and  respectively. 

The  corresponding  results  when  the  data  block  is  a  symmetric  extension 
of  an  Nj  by  N2  data  block  are  shown  in  Table  4.  Note  that  the  kth  scan  has 
period  ^  as  a  function  of  the  one-dimensional  scan  index,  so  that  only  ^ 
successive  values  for  the  kth  scan  need  be  shown.  It  will  be  noted  that, 
unlike  the  case  of  the  linear  congruential  scan  for  the  two-dimensional  DFT, 
each  scan  coordinate  changes  by  at  most  one  from  one  scanned  point  to  the  next. 


Table  4A  -  Scanning  Function  for  the  kth  Coordinate 
f  a  Two-Dimensional  DCT  Symmetrized  to  Odd 
Length  on  the  kth  Coordinate 


INDEX 


jk  (MODULO  Ny  jk  REEUCED  TO  ORIGINAL  DATA  BLOCK 


V1 

-(Nk-D 


2Nk"2 
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Table  4B  -  Scanning  Function  for  the  kch  Coordinate 
of  a  Two-Dimensional  DCT  Symmetrized  to  Even  Length 
on  the  kth  Coordinate.  (M^  =  2NjJ 

jk  (MODULO  Mr)  jR  REDUCED  TO  ORIGINAL  DATA  BLOCK 

0  0 

1  1 


INDEX 

0 

1 


2Nk-2 


The  vertical  and  horizontal  scan  functions  may  be  thought  of  as  sampled 
triangular  waveforms.  During  a  complete  sampling  of  the  extended  data  block, 
M2  complete  periods  of  the  triangular  waveform  of  period  occur,  and 
complete  periods  of  the  triangular  waveform  of  period  M2  occur. 

An  analog  version  of  the  required  scan  may  be  obtained  by  sampling  the 
integral  of  a  square  wave.  A  digital  version  may  be  obtained  from  an  up-down 
counter  with  appropriate  gating  to  reverse  the  count  direction  when  necessary. 
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CONCLUSIONS 


It  has  been  shown  that  two  different  types  of  two-dimensional  DCT  may 
be  implemented  as  one-dimensional  DCTs  of  an  appropriate  scan  of  the  data 
block.  For  a  hybrid  ODCT  by  ODCT,  a  single  complex  filter  of  length 
(2NJ-1) (2N2-1)  is  needed.  For  a  hybrid  ODCT  by  EDCT,  either  two  conplex 
filters  of  length  ^(2^-1)  or  a  single  complex  filter  of  length  (2^-1) (2N2)-1 
is  needed.  The  required  scan  on  each  coordinate  is  a  sampled  triangular  wave 
and  thus  may  be  easily  generated  and  has  no  abrupt  changes  of  the  coordinate 
values . 


C-13 


REFERENCES 


[1]  Means ,  R.  IV.,  H.  J.  IVhitehouse,  et  al,  Image  Transmission  via  Spread 
Sp_e.ct:rum  fechniques,  ARPA  Quarterly  Technical  Report,  March  1  -  June  1, 
1973,  Order  Number  2303,  Code  Number  3G10. 

[2]  Means,  R.  IV.,  J.  M.  Speiser,  H.  J.  IVhitehouse,  et  al,  Image  Transmission 
Via  Spread  Spectrum  Techniques.  ARPA  Quarterly  Technical  Report,  June  1  - 
October  1,  1973,  Order  Number  2303,  Code  Number  3G10. 

[3]  Ahmed,  N.,  T.  Natarajan,  and  K.  R.  Rao,  On  Image  Processing  and  a  Discrete 
Cosine  Transform,  to  appear  in  the  IEEE  Transactions  on  Electronic 
Computers. 

[4]  Speiser,  J.  M. ,  High  Speed  Serial  Access  Implementation  for  Discrete 
Cosine  Transforms.  NUC  TO  1265,  8  January  1974. 


C-14 


ON  THE  EQUIVALENCE  OF  ONE  AND  TWO  DIMENSIONAL  FOURIER  TRANSFORMS 

By  J.  W.  Bond 

ABSTRACT 

Two  dimensional  discrete  Fourier  transforms  can  be  calculated  by  ordering  an 
Nj  by  N2  array  of  data  and  calculating  a  one-dimensional  Fourier  transform, 
provided  N  j  and  N2  are  relatively  prime.  This  note  shows  how  any  isomorphism 
from  the  direct  sum  of  the  integers  modulo  N  j  and  N9  to  the  integers  modulo 
N 1 N2  can  be  used  to  relate  one  and  two  dimensional  discrete  Fourier  Transforms. 
Among  the  available  isomorphisms  it  is  always  possible  to  choose  one  that  has  either 
a  particularly  simple  input  scan  or  a  particularly  simple  output  scan.  Both  the  input 
and  output  scans  can  be  particularly  simple  only  when  Nj  and  N2  differ  by  one. 

These  simple  scans  consist  of  spiraling  along  a  discrete  torus.  Such  scans  can  be 
calculated  with  a  pair  of  counters  modulo  the  horizontal  and  vertical  block  size. 

INTRODUCTION 

Discrete  Fourier  transforms  have  been  of  considerable  interest  since  the  discovery  of 
algorithms,  known  as  Fast  Fourier  Transforms,  which  enable  the  rapid  calculation  of  dis¬ 
crete  Fourier  transforms  on  digital  computers.  R.  W.  Preisendorfer**  noticed  that  one¬ 
dimensional  Fourier  transforms  of  length  N  could  be  implemented  as  two  dimensional  N  ] 
by  N2  Fourier  transforms  when  N  ,  N9  =  N  with  Nj  and  N9  relatively  prime  and  an  iso¬ 
morphism  of  the  integers  modulo  N  is  used  onto  the  direct  sum  of  the  integers  module  Nj 
and  the  integers  module  N2. 

Jeff  Speiser  and  Harper  Whitehouse  realised  that  such  an  isomorphism  could  also  be 
used  to  implement  the  two-dimensional  discrete  Fourier  transform  as  a  one-dimensional 
transform.  They  suggested  a  particular  architecture  which  would  enable  the  calculation  of 
two-dimensional  Fourier  transforms  at  much  higher  rates  than  the  FFT  implementations 
which  are  usually  used  at  present.*  In  this  report  it  is  shown  that  either  the  input  scan  or 
output  scan  utilized  in  this  architecture  can  be  chosen  as  a  simple  diagonal  scan  on  the  torus. 

R.  W.  Means,  Speiser,  J.  A/.,  Whitehouse,  U.  J.,  el.  at,  Image  Transmission  via  Spread  Spectrum  Techniques  ARPA 
„  Quarterly  Technical  Report  June  I  -  October  I,  1973,  Order  Number  2303  Code  Number  3G1 0 
R.  W.  Preisendorfer,  Introduction  to  Fast  Fourier  Transforms,  Visibility  Lab,  University  of  California,  San  Diego, 
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Furthermore,  it  is  shown  that  both  input  and  output  scans  can  be  chosen  to  be  simple  only 
when  the  array  block  sizes  differ  by  one. 

THE  DIAGONAL  SCAN 

An  Nj  by  N2  scan  is  defined  by  specifying  an  Nj  by  N2  matrix  S  =  (nv),  where  {n-- 1  is 
n  |  o  <  n  <  NjN2  -  If.  The  result  of  a  scan  applied  to  an  Nj  by  N2  array  of  data  A  =  (ay) 

is  to  order  the  data  corresponding  to  nq.  For  example,  the  scan  (  ?  \  )  applied  to  the 

(a  b  c\  J  \j  I  5  / 

d  e  f  )  orders  the  data  a,  e,c,d,b,f. 

The  simplest  scan  of  interest  in  this  note  begins  with  0  in  the  upper  lefthand  corner 
and  numbers  successively  down  the  diagonal  according  to  the  rules  shown  in  Figure  1 .  Fig¬ 
ure  2  contains  examples  which  indicate  that  the  diagonal  scan  covers  an  N  j  by  N2  array  if 

and  only  if  N  j  and  N2  are  relatively  prime.  This  result  is  established  in  Corollary  1  of 
Proposition  l. 

DISCRETE  FOURIER  TRANSFORMS  AND  DIAGONAL  SCANS 

I 

Suppose  throughout  this  section  that  N  j  and  N2  are  relatively  prime.  The  top  corner 
of  the  diagonal  scan  of  an  N  j  by  N2  array  has  the  form 

/  0  u2N|\ 

\u1N2  1  / 

Later  we  will  show  that  the  integer  in  the  j  j ,  j2  spot  of  the  diagonal  scan  is  the  integer 
between  0  and  NjN2  -  1  congruent  modulo  NjN2  to  jj  Uj  N2  +j2  u2  Nj ,  Uj,  u2  integers. 
Given  real  or  complex  numbers  o  <jj  <Nj  -  1,  o  <j2  <N2  -  1 ,  let 

~  8Gi,j2)’wherep  =  ^  1  U1  ^2  +  j2  u2  N  j)  modulo  N  jN2.  Then  if 
NiN2-l 

Fq  =  ^  fp  exP  (-i  2 tt  pq/NjN2),  the  sequence 

p=o 

F0.  ^1  >  •  •  •  >  ^N|N2-1  *s  ca^ed  the  discrete  Fourier  transform  (D  F  T)  of  the  sequence 
fo’  ^1»  •  •  •  »  ^NiN2-1- 
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2 


\ 

START 


m+2 

^  .  m+3 


/° 


\ 


n+2 


1  n+3 


\ 


n 


n+1 


m 


CONTINUATION  OF  SCAN  WHEN 

(i,  j)  =  (Nj  -  1,  s) 

\ 


.  m+1 


CONTINUATION  OF  SCAN  WHEN 
(U)  =  i,N2  —  1) 


Inductive  definition  of  scan 

Assign  0  to  (0,0).  If  n  has  been  assigned  to  (i  J)  then  n+  1  is  assigned  to  (i  +  1  ,j+  1) 
with  i  +  1  reduced  to  zero  when  i  =  N  j  -  1  and  with  j  +  1  reduced  to  zero  when  j  =  N2  -  1 
provided  an  integer  has  not  already  been  assigned  to  (i  +  1 ,  j  +  1  )•  Note  the  indices  run 

from  0  to  Nj-1  or  N2~l  rather  than  from  1  to  Nj  or  N2. 

Note  also  that  if  k  is  assigned  to  (i,  j)  then  i  is  the  least  nonnegative  integer  congruent 

to  k  modulo  Nj  and  j  is  the  least  nonnegative  integer  congruent  k  modulo  N2. 


Figure  1 .  Definition  of  the  Diagonal  Scan. 
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Note  The  diagonal  scan  exhausts  an  N  j  by  N2  array  if  and  only  if  N  j  and  N2  are  relatively 
prime.  Indeed  the  diagonal  scan  naturally  defines  an  isomorphism  from  the  direct  sum  of 
integers  modulo  Nj  and  N2  to  the  integers  modulo  the  least  common  multiple  of  Nj  and 

n2. 

Figure  2.  Examples  of  the  Diagonal  Scan. 
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Let 


<1  -  k  j  N2  +  1<2  N| ,  kj ,  k9  integers, 


then 

Fk1N2  +  k2N1 
Nrl  N9-l 

=  E  E 

j]-0  j2-0 


fj ! u j N2  +  j2u2N!  exP  ((- i  2jr/N  1 N2)  (j ,  Uj N2  +  j2u2Nj )  (k ,  N2  +  k2Nj )) 


since  exp  (-i  27r/NjN9)  N|N9  -  1 ,  the  exponents  can  be  reduced  modulo  N]N9. 

Since  the  diagonal  scan  has  1  assigned  to  (1,1)  we  have  UjN9  +  u9N]  =  1  mod  NjN-,. 

Conclude  from  multiplication  by  from  this  equation  that  u]  =  N?  mod  NjN9 
and  by  N j  that  u2  nJ  sNj  mod  NjN^  Thus 

(jluiN2+j2u2Ni)(kiN2  +  k2Ni)  —  J 1 k  1 u  1  N9+J2k2u2N? 

=  G]kjN2  +  j2k2Nj)  mod  NjN9. 

Therefore 


k  1  n2  +  k2Nl 
N,-l  N7-I 

=  E  E  (fjlu1N2  +j9u9N, )  exP  ((- i  2tt/N , N9 )  (j  1  k , N2  +j9k9N,)) 
j  1 =0  J’2=0 


and  the  n 


G(0,0)  -  F0'  •  •  •  .  G(k[ ,  k2)  -  Fk|N2  +  k2N, 

is  II, e  two-dimensional  DFTofg(00),...,6(k|k2| . E(N|  .  ,,N 
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To  see  this  simply  note  that  the  previous  equation  when  written  in  terms  of  ■ 

,  ~  •  u  l  J?' 

LI 

Nrl  N2-l 

gG1J2)exP  t-i  2tt  Gikj/Nj  +j2k2/N2)]  . 

jpO  j2=0 

The  output  scan  associated  with  mapping  (kj,  k2)-»  (kjN2  +  k2Nj )  is  described  in  Fig¬ 
ure  3.  Examples  of  input  diagonal  scans  and  output  scans  are  illustrated  in  Figure  4;  the 
output  scan  appears  to  be  simple  when  N  j  and  N2  differ  by  1 .  That  this  is  the  only  case 
when  the  output  scan  is  simple  is  discussed  in  the  concluding  paragraphs  of  this  report. 

Note  that  the  output  scan  can  be  viewed  as  proceeding  on  diagonals  perpendicular  to  the 
input  scan  when  N  j  and  N2  differ  by  one. 

THE  GENERAL  THEORY 

Let  ZN  denote  the  additive  group  of  residue  classes  modulo  N.  An  element  of  ZN 
consists  of  all  integers  that  differ  by  a  multiple  of  N. 

! .  The  direct  sum  of  two  8rouPs  Znj  and  ZN2  is  the  grouP  written  ZN  j  +  ZN  consisting 

°f  pairs  of  elements  one  from  ZN ^  and  one  from  ZN  with  component  addition. 

The  group  ZNj  +  Z^  and  Z^^  are  isomorphic  (i.e.,  structurally  the  same)  if  and 
only  if  Nj  and  N2  are  relatively  prime.  The  clearest  way  to  see  this  is  to  exhibit  a  generic 
homomorphism  from  ZNj  +  ZN2  into  ZNj  jq2  and  show  that  some  of  the  homomorphisms 
are  onto  if  and  only  if  Nj  and  N2  are  relatively  prime. 

Any  homomorphism  \p  of  ZNj  +  ZN^  into  Zj^j^  is  completely  determined  by 
i//  (1,0)  and  i p  (0,  1 )  for  \p  (j  j  ,j2)  =  j  j  i p  ( 1 , 0)  +  j  2  \p  (0,  1 ).  Furthermore  N  j  \p  ( 1 , 0) 

|,  ^  (Nj  ,0)  -  \p  (0,0)  -  0  and  N2  \p  (0,1 )  =  \p  (0,N2)  =  \p  (0,0)  =  0  because  any  homomorphism 

must  take  (0,0)  onto  0.  If  we  interpret  the  residue  classes  \p  (1 ,0)'and  \P  (0,1)  as  integers 
between  0  and  N j N2  -  1 ,  then  Nj  i//  (1,0)  and  N2  i//  (0,  1)  must  be  divisible  by  N  j N2  so 
that  i//  (1 ,0)  =  ujN2  and  \p  (0,1)  =  u2Nj  for  some  integers  uj  and  u2  between,  respective¬ 
ly,  0  and  Nj  -  1  and0andN2-  1.  Therefore  ^  Oj , j2)  =  jiujN2  +j2u2Nj  forO^jj  <Nj 
-  1 , 0  <  j2  <  N2  -  1 .  The  term  ij,  will  be  onto  when  the  j  j  u  j  N2  +  j2u2N  \  reduce  Modulo 
I  N  j  N2  to  all  the  integers  between  0  and  N  j  N2  -  1 . 


f 
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If  the  input  scan  is  the  diagonal  scan 


INPUT  OUTPUT 
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4 

9 

14 

J 

Figure  4.  Examples  of  Input  Diagonal  Scans  and  Associated  Output  Scans 

If  we  reduce  the  elements  of  S  modulo  NjN2  we  will  get  all  integers  between  0  and 
NjN2-  1  if  and  only  if  there  exists  e  such  that  de=  1  mod  NjN2,  i.e.,  d  is  a  unit  in  the 
residue  class  modulo  NjN^  This  is  immediate  when  we  observe  all  the  elements  in  S  are  of 
the  form  kd,  k  an  integer.  If  kd  =  1  mod  NjN2  for  some  k,  then  every  integer  can  be  ob¬ 
tained  between  0  and  NjN2  -  1 ;  conversely,  if  1  is  not  of  this  form,  then  1  is  not  in  the 
image. 

PROPOSITION  1 

Let  ^  :  ZNj+N2  ZNjN2  be  a  homomorphism. 

Then  i p  has  the  form: 

^  ^  1  ’  j  1 )  =  JlulN2+  J2u2N  1  witfl  the  common  divisor  of  u j N2  and  u2N  j  in  the 
integers  is  a  unit  in  the  intergers  modulo  NjN2. 
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Proof 


We  have  shown  that  any  homomorphism  has  the  above  form,  but  we  have  not  shown 
that  ip  is  well  defined,  i.e.,  if  j j  -j',  =kN,  and  j2  -  j'2  =  k'N2  then  \p  (jj  J^)  -  \p  ( j' ,  ,j'9) 

=  k"  N1N2-  Bllt'  'p  O'l  J2)-  P  G'  1 , j'2)  =  j  1 11 1  n2  +  J2u2N I  "  (i'iulN2  +j'2u9Nl) 

=  (j'l  -  j'] )  u , N2  +  0*2  “  J‘2)  u2N  1  =  kNjUjNo  +  k'N2u2N]  =  (ku j  +  k'u?)  Nj No. 

Corollary  1 

|  +  's  isomorphic  to  ^  if  and  only  if  N  ]  and  No  are  relatively  prime. 

Proof 

If  d  >  1  divides  Nj  and  No  then  dk  =  0  mod  N  jNo  w:th  0  <  k  <  NjNo  -  1 .  Note  then 
that  if  dk'  =  1  mod  NjN2  for  some  k",  dk'  -  1  =  k"  NjNo  and  hence  dkk'  -  k  =  kk"  NjNo, 
i.e.,  k  N]N2-k  =  kk  N]N2  or  k  =  0  mod  Nj  No,  a  contradiction. 

Now  we  wish  to  work  out  the  relationship  between  scans  and  homomorphism s.  Sup¬ 
pose  \p  is  a  homomorphism  from  ZN^  +  ZN^  into  ZN  Nv 
Then 

iKO.O)  tKO.l) 

iKi.O)  ^(i.i) 

i/'fN,  - 1,0)  \KN,-  1,1) 

is  the  associated  scan. 

PROPOSITION  2 

The  diagonal  scan  defines  a  homomorphism  from  ZL  onto  Z^  +  ZN^  where  L  is  the 
least  common  multiple  of  N  j  and  N9. 

Proof 

Note  by  the  definition  of  the  diagonal  scan  of  an  N  ]  by  N-,  array  shown  in  Figure  1 
every  element  of  the  first  row  is  divisible  by  N  j  and  every  element  of  the  first  column  is 
divisible  by  N2.  The  assignment  of  integers  continues  until  the  bottom  corner  is  reached  so 
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that  the  next  spot  is  zero.  Then  if  we  let  L-l  denote  the  entry  in  the  bottom  corner  L  must 
be  congruent  to  0  for  the  scan  to  define  a  homomorphism.  Since  L  is  congruent  to  an  ele¬ 
ment  in  the  first  row  and  first  column  it  is  a  common  multiple  of  Nj  and  N2.  Therefore 
L>  the  least  common  multiple  of  N  j  and  N  2 . 

Next  consider  what  pairs  0  \ ,  j2)  can  be  assigned  integers  by  a  diagonal  scan.  Let  d  be 
the  greatest  common  divisor  of  N  j  and  N2.  Note  that  the  integers  reached  come  from  (k,  k) 
with  k  reduced  modulo  Nj  in  the  first  index  and  k  reduced  modulo  N2  in  the  second  index. 
Therefore  if  0  j ,  j2)  is  assigned  an  ,  ..uger,  we  have  (j  j  ,j2)  =  (k+k  ,N  j ,  k+k2N2)  for  some 
integers  k j  and  k2;  so  that  jj  -  j2  =  kjNj  -  k2N2  =  k3  d.  In  particular,  the  only  integers 
reached  in  the  first  row  (i.e.,  when  jj  =  0)  are  multiples  of  d  and  in  the  first  column  are 
multiples  of  d.  More  generally,  the  diagonal  only  fills  out  1/d  of  the  array.  Therefore  at 
most  1/d  of  the  locations  in  an  array  can  be  covered  by  the  scan.  Since  NjN2/d  is  the  least 
common  multiple  of  Nj  and  N2>  we  have  L  <  the  least  common  multiple  of  Nj  and  N2. 

Thus  L  is  the  least  common  multiple  of  Nj  and  N2. 

Next,  we  work  out  the  relationship  between  discrete  Fourier  transforms  and  the 
isomorphisms  we  have  been  constructing.  It  will  turn  out  that  the  mappings  under  discus¬ 
sion  must  be  isomorphisms. 

^  *  18  any  one  to  one  maPPing  of  zNj  +  zN2  °"to  zNjN2  then  ^  induces  a  mapping 
«//  from  the  functions  indexed  by  ZN  j  ^  to  those  indexed  by  ZN  +  ZN  by  defining 
<**0(k1,k2)  =  flMkl,k2).  2 

Let  DFT1  denote  the  one  dimensional  Fourier  transform  which  maps  a  function  in- 
dexed  by  ZL  into  functions  indexed  by  ZL.  In  particular 

L-l 

Fq  =  DFTi (f)  =  ^2  fP exp  ((_i  2n/L)  pq)- 
P=o 
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Let  DFT2  denote  the  two  dimensional  Fourier  transform  which  maps  a  function  g 
indexed  by  ^  +  Z^  into  a  function  indexed  by  Zj^  ^  +  Z^y  In  particular,  G  (k  j ,  k2> 


ll 


-  DFT-(k  M(S) 


Nrl  N->-l 

■El  g(j  i  exp  l-i  27r  (j  j k  |  /N  j  +  j^k^/No)] 
j  j  =0  jo~0 


Nrl  No-1 

•i :z 

j  j  — 0  j2-^ 


g  exp  ((— i  27t/N  1  N->)  (k  ij  1N1  +  k->j-)N  ])) 

(j  1  J2^> 


The  relationship  between  DFT1  arid  DFT2  is  summarized  in  the  following  proposition. 

PROPOSITION  3 

a)  If  \p  and  y  are  on-to-one  maps  from  Z^  +  Z^  onto  Z^  v]0  such  that  the  diagram 


commutes  then; 

1)  N[  and  N2  are  relatively  prime  and 

2)  i p  and  <p  are  isomorphisms  of  ZN  j  +  ZN2  onto  ZNjN2- 

b)  If  tp  is  any  isomorphism  of  Zj^  +  Z^  onto  Z^^  then  there  exists  an  iso¬ 
morphism  ip  so  that  the  diagram 


commutes. 
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n 

i . 


L 


c)  If  ^  0 1 » j2)  =  Jl uiN2  +  j2u2Nj  then 

^  (kl >  k2)  =  klViN2  +  k2v2Nj  withu,v]N2  +  u2v2N1  =  ]. 

Note  that  the  commutativity  of  the  above  diagrams  is  simply  the  mathematical  way  to 

:*^r;rrr and  *; ei,her  dft2  -  be  -p*— « 

|DFT2[i*(m)  (g)l('°rDFT  implemented  by  DFT?,  namely  F  = 

N  In  '  shows  that  these  implementations  are  only  possible  when 

f,  2  are  re'“,Vely  Prime  a"d  ,he  indices  are  tsomorphisms  of  the  group 

of  residue  classes  under  addition.  P 

Before  proving  the  proposition  we  prove  the  following  Lemmas. 


Lemma  1 
If 


DFTJ 


DFT2 


then  i//  0" i > J*2 )  V3  (k]k2)  =  J 1 k i N2  +j2k2Nj  mod  NjN2  . 
Proof 

That  the  diagram  commutes  means  that 
G  =  DFT2  (i//*  0  =  [DFT1  (f)] 


To  calculate  DFT?  (*•  f)  =  *>  [DFT*  (0)  consider  the  component  mapping 

DFT2(k„k2)  =  G(klik2, 


Npl  N2-l 

ZJ  £  (^’  °0i.j2)exP  U  2"/NiN2)G,k|N2  +j2k2N,) 


Nrl  N2-l 

^  f^(il’j2)eXp("l27r/NlN2)<J]k]N2+j2k2N1) 
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To  calculate  <p  [DFt'  (f)]  let  G  =  <p*  F.  Then 

n,n2-i 

c(k,,k2)  =  F^k,,k2)  =  ^2  fkexPH27r/N,N2)^(k1,k2)p 

p=0 

N  j-1  N2-l 

ty(j  1  Jo)  exp  (-i  2?f/N  1n2^  *  (k  1 >  k2)  'P  (i  1 J2)  • 

j,=o  j2=0 

The  two  expressions  are  the  same  only  if  y>  (kj ,  k2)  1//  (j)  J2)  =  j  ] k  jN2  +  j2k2N  j  mod 
NjN2  for  all  j,  J2,k,,  k2. 

Lemma  2 

If  0]  J2)<£  (k  1 ,  k2)  =  (j  1  k  1 N2  +  j2k2N,)mod  NjN2  for  all  integers  j  j ,  j2,  k  j ,  k2 
then  \p  and  y>  one  to  one  mappings  of  ^  +  Zj^  onto  Z^  implies  that  \p  and  are 
isomorphisms. 

Proof 

For  (j'i  J2)  such  that  (j'|  Jo)  =  ' '  'P  (kj  +  k'j ,  k2  +  k'2)  =j'j  (k  j  +  k'j)  N2  +  j2  (k2 
+  k2)  N|  =  (j'jk]N2  +  j2k2N  j)  +  <j  j k  j  N 2  +  j2k'2N])  =  ^  (kf  ,k2)  +  (k',  ,jk'2)  mod  NjN2> 

Likewise  for  (k'j  k'7)  such  that  <p  (k'j ,  k'-J  =  1  it  follows  that  \p  is  a  homomorphism. 
Lemmas  1  and  2  complete  the  proof  of  part  (a)  of  proposition  3.  To  prove  b,  note 
that  if  \p  (j  ]  J2)  =  J 1  u1^2  +  -buoN  1  ’  the11  f*ie  existence  of  \p  is  equivalent  to  the  existence 
of  V]  and  v2  such  that  Ck  j ,  k-i)  =  k  jvjNt  +  k7V2N  j  and 

(j j u  1 N 2  +  j^U'tN])  (k]V|N7  +  k^VoN])  =  (j j k j N->  +  j2k]Nj)  mod  NjN2  . 

Let  vj  and  v2  satisfy  u  ]  v  |  K 1  +  U7V7N  j  =  1  MOD  N  jN7  which  also  has  a  solution 
since  U]N2  and  u2N  ]  have  greatest  common  divisor  in  the  integers  which  is  a  unit  in 
Zn  ] Mo‘  ^  v'l  anc^  v2  solves  uiv']N2  +  l^v^Nj  =  d,  d  the  greatest  common  divisor  of  UjN2 
and  u2N  ] ,  then  vj  =  e  v'j  mod  N  jN2  and  v->  =  e  v^  mod  N]  N7  where  de  =  1  mod  NjN2 
will  provide  a  solution  to  U]V|N2  +  u2v2N ]  =  1  mod  N  jN2. 
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From  ujvjN2  +  U2V2NJ  =  1  modNjN2  it  follows  from  multiplication  by  N  j  and  N2 
that  U2V2N j  =  N  j  mod  N jN2  and  =  N2  mod  N These  congruences  allow  us 

to  calculate  modulo  N  j  N2 

GiuiN2+j2u2NlHklvlN2  +  k2v2Nl)  =  jlklulvlN22+j2k2u2v2Nl2 
=  j ik jN2  +j2k2Nl  . 

which  completes  the  proof  of  proposition  3. 

There  are  two  ratural  symmetries  in  the  scans  associated  with  an  isomorphism.  These 
arise  because 

^  (Jl  +  lJ2+  O-'/'  O1J2)  =  U1N2  +  U2N1 

and 

^  Gl  +  1 J2  -  D-  Gl  J2)  =  u1n2  _u2N1 
are  independent  of  (j  j ,  )2). 

The  simplest  possible  scans  occur  when  one  of  these  two  differences  is  one. 

Note  that  if  <//  (j \  +  1 J2  +  1)  -  O1J2)  =  *  then  vi  =  1>  v2  =  1  and  </>  (kj,  IC2) 

=  k  jN2  +  k2N  j  defines  the  output  scan  so  that  the  output  scan  is  only  simple  when  N  j  and 
N2  differ  by  one  since  \p  G 1  +  1 J2  -  1 )  -  ^  0|  >  ij)  =  ±  1  •  This  results  because  for  all  cases 
of  interest  N  j  +  N2  is  small  composed  with  Nj^.  U3N2  -  U2N  =  1  then  vj  =  1  V2  =  -  1 
provides  a  solution  and  Gj .  =  ^1^2  “  J2^1>  anc*  here  tke  on^  t'me  that  ^  Oj  +  1> 

]2  +  1 )  -  ^  G 1 J2)  =  ±  1  is  when  N  j  and  N2  differ  by  one.  Therefore,  the  input  and  output 
scans  are  simple  only  when  N  j  and  N2  are  both  relatively  prime  and  differ  by  one. 


From  u  jv  jN2  +  u^V2N  |  =  1  mod  NjN2  it  follows  from  multiplication  by  N  |  and  N2 
that  U2V2N  j  =  N 1  mod  and  Ujvj^  =  N2  mod  Nj^.  These  congruences  allow  us 
to  calculate  modulo  N  j  N2 

0'iUjN2+j2u2Nl)(kiV]N2  +  k2V2Nj)  =  jjkjU|V1N22  +J2k2u2v2Nl2 

=  -i  1  k  1 N2  +  j2k2N !  , 

which  completes  the  proof  of  proposition  3. 

There  are  two  natural  symmetries  in  the  scans  associated  with  an  isomorphism.  These 
arise  because 

<//  G'i  +  1 J2+  l)-’/'  G1J2)  =  U1N2  +  U2N1 

and 

^  Oi  +  1,  j2  “  0“  t  CJ1J2)  =  U1N2“U2N1 

are  independent  of  (j  j ,  j^). 

The  simplest  possible  scans  occur  when  one  of  these  two  differences  is  one. 

Note  that  if  $  (jj  +  1 , j’2  +  1)  -  Gj , =  ^  then  vj  =  1,  V2  =  1  and  ip  (kj,  k2> 

=  k  jN2  +  k2N  j  defines  the  output  scan  so  that  the  output  scan  is  only  simple  when  N  j  and 
N2  differ  by  one  since  i|/Qj  +  I,j2-t)-^0]>j2^  =  ±  1  •  This  results  because  for  all  cases 
of  interest  N  j  +  N2  is  small  composed  with  N U1N2  -  U2N  =  1  then  vj  =  1  V2  =  -  1 
provides  a  solution  and  (jj  J2) =  j  1 N2  - J2^i  >  and  here  the  only  time  that  1//  Oj  +  1, 

)2  +  1)  -  ^  (Ji  J2)  =  -  1  is  when  N  j  and  N2  differ  by  one.  Therefore,  the  input  and  output 
scans  are  simple  only  when  N  j  and  N2  are  both  relatively  prime  and  differ  by  one. 
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ABSTRACT 


A  new  sign'll  processing  sensor  is  possible  using  a  combination  of  charge 
transfer  devices  and  signal  processing  concepts  which  simultaneously  measures 
the  incident  optical  signal  and  performs  a  linear  transformation  upon  that 
signal.  This  paper  will  discuss  possible  implementations  and  the  types  of 
transforms  which  may  be  performed.  It  appears  possible,  at  this  time,  to 
perform  the  Discrete  Fourier  Transform  in  a  linear  imaging  device. 

Extension  of  the  concept  to  a  two-dimensional  array  may  be  feasible. 
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INTRODUCTION 


t 

i 


In  signal  processing  optical  images  often  constitute  one  of  the  many  forms  which  signals 
may  take.  These  images  are  typically  processed  either  optically  by  means  of  moving  trans¬ 
parencies  or  optical  diffraction  cells  through  which  the  image  passes  prior  to  its  detection  by 
a  photodetector,  or  the  optical  signals  are  converted  to  electrical  signals  by  means  of  image 
sensors  and  these  electrical  signals  are  subsequently  processed  electronically. 

In  this  paper  a  new  optical  sensor  is  proposed  which  combines  the  functions  of  signal 
processor  and  image  sensor  in  a  single  photosensitive  silicon  Charge  Coupled  Device  (CCD).  For 
simplicity  of  description  and  ease  of  fabrication  a  one  dimensional  signal  processing  sensor 
is  described.  However,  in  many  of  its  applications  a  two  dimensional  array  of  one  dimen¬ 
sional  sensors  would  be  required.  In  principle,  these  two  dimensional  arrays  can  be  formed 
by  the  juxtaposition  of  a  number  of  one  dimensional  arrays  on  a  single  silicon  chip  with 
common  clocks  to  all  CCD  registers. 

ARCHITECTURE 

Both  500  X  1  and  100  X  100  photosensitive  silicon  CCD  arrays  have  been  commer¬ 
cially  fabricated'using  buried  channel  techniques  with  small  charge  transfer  inefficiencies  at 
clock  rates  in  excess  of  1  MHz.  Using  similar  techniques,  it  appears  feasible  to  simultaneously 
control  the  amount  of  photogenerated  charge  and  its  transverse  transfer  to  the  CCD  shift 

registers.  Successive  iteration  of  the  charge  generation  and  transfer  corresponds  to  a  weighted 
superposition  of  lagged  versions  of  the  image  and  thus  corresponds  to  a  discrete  convolution 
of  the  image  as  sampled  by  the  spatial  structure  of  the  sensor  with  the  time  function  repre¬ 
sented  by  the  transfer  signal.  Thus,  in  a  single  structure,  are  combined  the  functions  of  image 
sensing,  spatial  to  temporal  multiplexing,  and  convolution.  A  block  diagram  of  a  signal  proc¬ 
essing  array  is  shown  in  Fig.  1 .  The  discrete  nature  of  the  CCD  image  sensor  is  indicated  by 
the  boxes  indicated  by  “array”,  the  controlled  transfer  of  the  generated  charge  is  indicated 
by  “multiplication”,  and  the  CCD  registers  are  indicated  by  “delay”.  The  serial  output  of 
the  signal  processing  array  thus  occurs  at  the  clock  rate  of  the  CCD  registers  which  may  not 
be  regular. 

1.  I'airchild  Camera  &  Instrument  Corp.;  Syosset,  New  York. 
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Since  convolution  and  correlation  are  both  basic  to  signal  processing,  their  interpre¬ 
tation  as  discrete  spatial  operations  will  be  reviewed.  The  cross  convolution  and  correlation 
of  two  continuous  functions  are  given  by  equations  (1 )  and  (2)  respectively: 


(a)  convolution  [f  *  g]  (x)  =  [  I'(tj)  g(x-r?)  dr? 


(1) 


(b) 


correlation  [f®g]  ( x)  -  J  f(r?)  gfx+rj)  dr? 


(2) 


The  corresponding  relationships  for  discrete  functions  may  be  found  by  replacing 
the  integrals  by  products  as  given  in  equations  (3)  and  (4): 


(a) 

convolution 

=ZFngn-k 

(3) 

k 

(b) 

correlation 

[f  ®  gl  n  -'S  ln  Sn+k 

(4) 

The  value  of  the  discrete  convolution  or  correlation  may  be  viewed  as  the  coefficient  of  Zn 
in  the  polynomial  products  3'  and  4'  respectively. 

Thus 


(3') 


and 


f®8,nZ"=^rczc^|8mz^ 


(4') 


As  an  example  the  convolution  and  correlation  of  the  binary  sequence  1 ,1 ,1 .1 ,  with  a  non 
negative  sequence  1 , 1 ,1 ,1  is  given  below  where  overbar  denotes  negation. 
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(a)  convolution  1,2,1,2,1,0  1 

(b)  correlation  1,0, 1,2, 1,2,1 

Due  to  the  representation  of  the  sampled  variable  by  the  amount  of  minority  charge 
in  the  potential  well  induced  in  the  silicon,  only  numbers  of  a  fixed  sign  can  be  represented. 
This  presents  a  problem  when  it  is  desired  to  convolve  the  data  with  a  sequence  having  both 
positive  and  negative  terms.  This  difficulty  does  no,  arise  in  ordinary  image  sensors  since 
simple  sensing  of  the  light  intensity  requires  only  numbers  or  the  same  sign.  Unfortunately, 
the  solution  which  is  used  in  signal  processing  applications  of  CCD  of  adding  a  fixed  bias  of 
half  the  maximum  value  of  the  input  signal  is  difficult  for  a  signal  processing  image  sensor, 
since  the  bias  signal  would  then  need  to  be  optical  at  half  the  intensity  of  the  maximum 
optical  input.  However,  by  designing  an  image  sensor  with  two  CCD  registers  and  bidirec¬ 
tional  charge  transfer,  it  is  possible  to  sense  the  difference  between  the  charge  in  the  two 
registers,  and  m  this  way,  represent  both  positive  and  negative  numbers. 


Two  applications  of  this  proposed  signal  processing  image  sensor  are  considered  as 
examples  of  its  potential  use.  These  are  real  time  cross  correlation  or  filtering,  and  transform 
image  encoding.  Cross-correlation  uses  spatially-uniform  time-varying  illumination.  Trans¬ 
form  image  encoding  uses  a  spatially-varying  illumination  which  is  constant  during  this 
processing  interval. 

An  optical  cross  correlator  or  optical  filter  is  shown  diagrammatically  in  Fig.  2.  The 

input  signal  f  modulates  the  light  output  from  a  photo  diode  one  focal  distance  in  front  of  a 

converging  lens.  The  collimated  light  then  falls  on  the  signal  processing  image  sensor  whose 

transfer  gates  are  controlled  by  the  signal  g.  The  output  from  the  sensor  will  then  be  the 

cross  correlation  of  the  input  signal  f  with  the  transfer  gate  signal  g,  and  will  be  available  in 

real  time  at  the  same  clock  rate  as  the  input  signal  f.  This  is  an  improvement  proportional 

to  log2N  in  the  rate  of  computation  over  evaluation  by  “in  place"  digital  computation  using 

the  “fast  convolution”  algorithm  on  a  computer  with  the  same  cycle  time  as  the  input  signal 
sample  rate. 

A  transform  image  encoder  is  shown  diagrammatically  in  Fig.  3.  An  input  object  is 
at  infinity  before  a  converging  lens  of  focal  length  d.  An  image  is  formed  a  distance  d  behind 
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the  lens  on  a  signal  processing  image  sensor.  If  a  signal  g  is  applied  to  the  transfer  gates  of 
the  Signal  Processing  Image  Sensor  (SP1S),  the  cross  correlation  of  the  image  and  the  signal 
g  will  be  obtained  as  an  output,  provided  the  image  is  stationary  for  the  duration  of  the 
signal  f. 


Proposed  Solution 

The  signal  processing  concepts  presented  and  illustrated  above  may  be  implemented 
by  an  idealized  structure  shown  in  Fig.  4.  The  suggested  device  configuration  will  allow  the 
controlled  routing  and  algebraic  summation  of  analog  signals.  The  structure  consists  primar¬ 
ily  of: 

a.  a  sensing  array  of  MIS  capacitors,  called  photo-gates, 

b.  two  CCD  shift  registers  on  either  side  of  the  line  sensing  devices, 

c.  a  dual  transfer  gate  system  to  control  the  direction  in  which  the  signal  charge 
flows  out  from  the  sensor  area. 

When  an  image  is  projected  on  the  sensing  elements,  electron-hole  pairs  are  generated.  The 
electrons  (in  a  p-type  semiconductor  substrate)  will  be  normally  recombineu  within  a  diffu¬ 
sion  length  from  their  generation  sites  in  the  deflection  regions  formed  under  the  photogates. 
By  independently  adjusting  the  surface  potential  under  the  transfer  gates,  one  is  able  to  limit 
the  diffusion  and  to  transfer  the  stored  charges  from  under  the  sensors  (e.g.,  photogates) 
onto  the  desired  CCD  register  for  subsequent  mathematical  operations. 

Binary  Convolution 

The  implementation  of  a  discrete  convolution  will  now  be  considered  in  detail.  The 
two  sequences  can  be  easily  represented  (1 )  as  the  input  signal  ej(t)  in  the  form  of  charge 
packets  generated  and  stored  under  each  sensing  element  and  (2)  as  a  digital  pattern  of 
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narrow  pulses  Wk  applied  to  the  transfer  gates.  The  convolution  of  these  two  functions  or 
sequences  yields  a  third  one  which  is  mathematically  described  as  the  superposition  of  the 
effect  of  past  input  excitations  to  account  for  the  present  output.  If  one  assumes  that  the 
device  under  consideration  has  not  reached  saturation,  i.e.,  the  different  potent, al  wells  are 
not  completely  filled,  the  output  is  expressed  directly  in  terms  of  an  explicit  summing  opera¬ 
tion  upon  the  input.  Thus,  the  present  value  of  the  output  is  a  weighted  sum  over  the  history 
of  the  input  ej(t). 

The  explicit  input-output  relationship  is: 
eo(0  =  [woei(t)  +W,ei(t-T)  +  W2ei(t-2T)  +  . . .] 

or  in  other  notation 
N 

e0(t)  =  ^  Wkej(t-kT) 
k=o 

where  WR  is  the  weighting  function  associated  with  the  kth  position  of  the  pulse  in  the  tim¬ 
ing  sequence.  In  the  present  case,  the  weights  WR  are  treated  as  binary  numbers  +  or  -  1 . 

The  output  is  thus  a  weighted  sum  of  the  image  samples  with  coefficients  +  or  -  1 . 

The  fundamental  building  block  shown  in  Fig.  4  provides  an  output  signal  e0(t)  as 
expected  from  equation  ( 1 )  -  that  is  the  product  of  the  sensing  element  signal  and  a  param¬ 
eter  set  by  the  digital  timing  sequence  applied  to  the  transfer  gates  and  is  algebraically  summed 
at  the  output  of  the  CCD  shift  registers  to  account  for  past  excitations.  The  convolution 
operation  is  thus  seen  to  be  resulting  from  a  series  of  shifting,  multiplication  by  a  weighted 
constant  and  summation.  This  is  illustrated  in  Figure  5  which  shows  how  an  image  is  con¬ 
volved  with  a  digital  pattern.  The  four  images  a,  b,  c,  and  d  which  constitute  the  input  signal 
ej(t)  illuminate  four  adjacent  photogates  as  depicted  in  Figure  6(a).  The  result  of 
applying  the  weighting  sequence  WR  described  by  the  digital  pattern  1 , 1 , 1 , 1  is  shown 
in  Fig.  6(b).  The  weighting  sequence  is  decomposed  into  two  timing  sequences  to  be 
applied  appropriately  to  the  transfer  gates.  The  resulting  output  convolutions  or  cor¬ 
relations  are  shown  in  Fig.  6(c)  for  two  assumed  illuminations  a=b=c=d=l  and 
a=b=d=l ,  c=0. 
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Amplitude-weighted  Convolution 


The  above  example  dealt  with  binary  patterns  which  were  defined  by  the  timing  dia¬ 
gram  relative  to  both  transfer  gates.  Integration  time  or  pulse  position  affect  the  number  of 
generated  carriers  which  are  stored  in  the  sensing  MIS  elements.  Pulse  width  and  pulse  ampli¬ 
tude  control  the  partial  or  total  transfer  of  the  carriers  from  the  sensor  potential  wells  into 
the  CCD  shift  registers.  Thus,  the  mechanism  by  which  the  charges  spill  over  into  the  CCD 
shift  registers  for  signal  processing  is  controlled  by  one  or  more  of  these  parameters:  integra¬ 
tion  time,  pulse  width  and  pulse  amplitude: 

a.  The  integration  time  is  the  time  needed  to  generate  sufficient  number  of  carriers 
to  adequately  characterize  the  object  (signal)  to  be  measured.  During  integration  lime,  the 
photogate  is  energized,  thus  forming  a  depletion  layer  under  it  to  attract,  capture  and  hold 
the  photon  generated  minority  carriers.  The  number  of  these  carriers  is  proportional  to  the 
integration  time,  the  intensity  is  the  signal  and  the  illuminated  area. 

b.  The  surface  potential  established  under  the  transfer  gate  electrode  is  proportional 
to  the  pulse  amplitude  applied  to  the  gate.  Narrowing  the  pulse  width  with  or  without  an 
amplitude  reduction  results  in  restricting  the  flow  of  charges  out  from  their  storage  area. 

Widening  the  pulse  and/or  increasing  the  pulse  amplitude  has  the  effects  of  totally  emptying 
the  MIS  depletion  regions. 

A  crucial  deficiency  appears  in  mechanism  (b)  when  compared  to  the  first  one  as  it  leaves  a 
fraction  of  the  signal  in  the  sensor  area.  This  unused  portion  of  signal  when  added  to  subse¬ 
quent  signals,  will  interfere  with  the  desired  device  characteristics.  There  is  however  a  means 
to  remove  the  lelt-over  charges:  by  negatively  biasing  the  photogates  to  bring  them  into  accu¬ 
mulation.  During  the  process  of  accumulation,  majority  carriers  (holes)  are  drawn  to  the 
WSjO-,  interface  to  be  recombined  with  the  signal  carriers  (electrons).  The  mechanism  of 

recombination  is  however  slow  in  a  high  carrier  lifetime  material  as  is  the  usual  case  for  charge 
transfer. 

Mechanism  (a),  when  implemented  yields  a  more  efficient  operation  as  it  transfers  all 
available  charges  to  the  processing  CCD’s.  Assuming  that  the  saturation  level  of  the  potential 
under  the  photogates  is  not  reached,  one  is  able  to  adjust  the  fraction  of  the  well  to  be  filled 
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by  controlling  the  integration  time,  i.e.  by  adjusting  the  pulse  position  between  points  tj  and 
t2  of  Figure  7.  Subdividing  the  integration  time  into  “n”  equal  time  intervals,  the  signal 
ei(t)  can  be  measured  down  to  “n”  levels.  The  error  will  be  commensurate  with  the 
proportion  of  carriers  generated  during  the  dead  time  from  tj  to  t2  which  should 
therefore  be  at  most  one  hundreth  of  the  integration  time  in  order  to  provide  tap 
weights  accurate  to  one  percent.  The  emphasis  is  thus  on  pulse  position  variation 
to  control  the  level  of  the  sensed  signal  down  from  its  maximum  amplitude.  The 
four  images  a,b,c,and  d  which  constitute  the  input  signal  ej(t)  illuminate  four  ad¬ 
jacent  photogates  as  depicted  in  Fig.  8(a).  The  result  of  applying  the  weighting 
sequence  Wk  described  by  the  digital  pattern  1  is  shown  in  Fig.  8(b). 

The  weighting  sequence  is  decomposed  into  two  timing  sequences  to  be  applied 
appropriately  to  the  transfer  gates.  The  resulting  output  is  shown  in  Fig.  8(c)  for 
the  assumed  illumination  a=b=c=d=l . 
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(a)  IDEALIZED  CONFIGURATION  OF  THE  DEVICE 
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